-- Logs begin at Fri 2022-06-03 14:35:12 UTC, end at Fri 2022-06-03 15:45:16 UTC. -- Jun 03 15:10:48.720574 dv-cinder-831247-16-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 03 15:10:48.730968 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 3 15:10:48 2022] *** Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: nodename: dv-cinder-831247-16-iscsi Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: machine: x86_64 Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: clock source: unix Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: pcre jit disabled Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: detected number of CPU cores: 4 Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: current working directory: / Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: detected binary path: /usr/bin/uwsgi-core Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: your processes number limit is 47680 Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: your memory page size is 4096 bytes Jun 03 15:10:48.733089 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: detected max file descriptor number: 2048 Jun 03 15:10:48.733824 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: lock engine: pthread robust mutexes Jun 03 15:10:48.733824 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: thunder lock: enabled Jun 03 15:10:48.733824 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 03 15:10:48.733824 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 03 15:10:48.781495 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: Python main interpreter initialized at 0x5556ab4cf9c0 Jun 03 15:10:48.781495 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: python threads support enabled Jun 03 15:10:48.781495 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: your server socket listen backlog is limited to 100 connections Jun 03 15:10:48.781495 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: your mercy for graceful operations on workers is 90 seconds Jun 03 15:10:48.783060 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: mapped 403077 bytes (393 KB) for 2 cores Jun 03 15:10:48.783060 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: *** Operational MODE: preforking *** Jun 03 15:10:48.783060 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: *** uWSGI is running in multiple interpreter mode *** Jun 03 15:10:48.783060 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: spawned uWSGI master process (pid: 80211) Jun 03 15:10:48.783068 dv-cinder-831247-16-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 03 15:10:48.785925 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: spawned uWSGI worker 1 (pid: 80221, cores: 1) Jun 03 15:10:48.785925 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: spawned uWSGI worker 2 (pid: 80222, cores: 1) Jun 03 15:10:48.785925 dv-cinder-831247-16-iscsi devstack@keystone.service[80211]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 03 15:10:54.585012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 15:10:54.585012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: from cryptography.utils import int_from_bytes Jun 03 15:10:54.585817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 15:10:54.585817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: from cryptography.utils import int_from_bytes Jun 03 15:10:54.586124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 15:10:54.586124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: from cryptography.utils import int_from_bytes Jun 03 15:10:54.586608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 03 15:10:54.586608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: from cryptography.utils import int_from_bytes Jun 03 15:10:54.636001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 03 15:10:54.636195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 03 15:10:54.636369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] command line args: [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 03 15:10:54.636574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 03 15:10:54.636669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 03 15:10:54.636770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 03 15:10:54.636856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ================================================================================ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 03 15:10:54.636962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] command line args: [] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 03 15:10:54.637041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] admin_token = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.637232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 03 15:10:54.637332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config_dir = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.637406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ================================================================================ {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 03 15:10:54.637478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.637599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] config_source = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.637672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] admin_token = **** {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.637784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] debug = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.637961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] config_dir = [] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.638104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.638348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.638348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] config_source = [] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.638583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.638701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] debug = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.638810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.638918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639201 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639201 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_config_append = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_dir = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639908 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.639981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] list_limit = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.640085 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] log_config_append = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.640159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.640265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.640383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.640498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] log_dir = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.640604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] log_file = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.640679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.640838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.640908 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.641017 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.641143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.641212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.641352 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.641449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.641579 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.641695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.641837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.641939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.642043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.642163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.642251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.642379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.642559 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.642656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.642755 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.642842 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.642922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.643128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.643254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.643357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.643468 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.643566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.643768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.643886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.93/identity {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.644008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] publish_errors = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.644119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.644242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.644469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.644558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.93/identity {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.644662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.644808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] publish_errors = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.644903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.645000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.645096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.645206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.645348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.645428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] standard_threads = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.645539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.645632 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.645772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.645857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.645930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.646054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] standard_threads = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.646126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_journal = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.646222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.646312 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_json = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.646427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_stderr = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.646509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.646627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] use_syslog = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.646707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.646825 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] use_journal = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.646913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.646987 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] use_json = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.647107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.647187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] use_stderr = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.647289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] use_syslog = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.647366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.647517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 03 15:10:54.647587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.647774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.647858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.648033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.648130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.648248 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.648323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.648489 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.648567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.648735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.648893 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.648969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.649049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.649255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.649366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.649444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.649597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.649697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.649758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.650227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.650350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.650446 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.650576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.650576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.650776 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.650880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.650976 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.651070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.651173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.651173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.651362 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.651457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.651552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.651645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.651750 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.651846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.651941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.652035 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.652136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.652258 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.652379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.652482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.652578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.652731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.652841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.652989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.653100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.653221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.653330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.653452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.653562 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.653689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.653809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.653913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.654013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.654130 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.654232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.654364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.654567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.654689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.654786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.654892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.654995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.655109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.655181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.655310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.655378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.655553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.655620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.655786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.655853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.656019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.656084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.656253 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.656320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.656481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.656554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.656730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.656803 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.656968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.657036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.657239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.657319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.657422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.657503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.657651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.657728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.657879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.658790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.658909 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.659018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.659018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.659205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.659331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.659414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.659529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.659603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.659603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.659776 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.659902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.659979 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.660123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.660209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.660329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.660442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.660526 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.660642 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.660723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.660825 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.660902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.661017 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.661100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.661208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.661322 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.661436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.661512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.661626 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.661710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.661792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.661909 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.661993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.662113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.662233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.662316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.662445 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.662523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.662654 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.662743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.662861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.662975 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.663050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.663173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.663251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.663365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.663441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.663556 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.663631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.663741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.663821 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.663943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.664019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.664134 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.664210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.664284 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.664356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.664477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.664563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.664696 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.664781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.664928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.665011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.665176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.665261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.665414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.665502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.665682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.665763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.665872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.665951 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.666106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.666195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.666630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.666718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.666836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.666943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.667071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.667167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.667304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.667383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.667557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.667634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.667794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.667883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.668029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.668112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.668265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.668343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.668499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.668585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.668730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.668824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.668960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.669039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.669190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.669269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.669415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.669503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.669710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.669843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.669977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.670114 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.670218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.670325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.670430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.670552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.670740 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.670856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.671520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.672036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.672144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.672241 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.672338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.672927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.673024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.673129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.external = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.673227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.673324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.673419 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.673514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.673611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] auth.external = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.673743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.673889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.674023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.password = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.674140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.674257 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] auth.token = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.674372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.674487 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.674662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] auth.password = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.674792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.674910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] auth.token = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.675027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.675142 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.675257 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.675379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.675496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.675613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.675728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.675923 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.676046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.676171 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.676288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.676403 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.676518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.676633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.676754 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.676878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] credential.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.676995 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.677111 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.677226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.677844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.677933 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678777 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.678944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679172 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.679963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.680035 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] federation.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.680107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.680223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.680300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.680498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.680578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.680652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.680724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.680796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.680868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.681048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.681131 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.681321 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.681478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.681570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.681650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.681755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.681854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.682007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.682152 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.682270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.682351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.682476 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.682627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.682740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.682819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.682965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.683049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.683212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.683295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.683453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.683540 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.683686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.683777 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.683928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.684011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.684159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.684267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.684394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.684476 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.684640 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.684721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.684874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.684954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.685104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.685191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.685349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.685429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.685602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.685714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.685850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.686011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.686091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.686312 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.686468 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.686627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.686762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.686904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.687035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.687163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.687293 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.687420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.687548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.687695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.687824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.687951 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.688141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.688795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.688964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.689056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.689133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.689205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.689265 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.689374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.689453 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.689453 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.689597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.689695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.689775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.689945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.690025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.690185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.690267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.690436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.690516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.690709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.690789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.690940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.691021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.691180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.691261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.691411 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.691490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.691639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.691725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.691876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.691956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.692103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.692183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.692342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.692421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.692576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.692654 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.692817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.692896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.693052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.693132 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.693282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.693360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.693514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.693594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.693763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.693843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.693994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.694074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.694221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.694300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.694458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.694557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.694731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.694810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.694962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.695040 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.695195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.695275 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.695429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.695595 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.695675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.695835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.695923 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.696110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.696264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.696352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.696497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.696578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.696744 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.696826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.696992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.697072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.697238 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.697317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.697473 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.697554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.697703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.697781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.697933 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.698019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.698166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.698245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.698403 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.698483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.698656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.698756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.698904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.698984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.699149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.699229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.699378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.699464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.699628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.699708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.699868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.699947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.700094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.700180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.700334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.700412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.700568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.700648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.700802 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.700889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.701036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.701116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.701268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.701348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.701510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.701597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.701740 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.701818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.701977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.702055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.702214 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.702300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.702444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.702523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.702711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.702791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.702948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.703039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.703187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.703266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.703421 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.703501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.703685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.703843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.703921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.703993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.704140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.704221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.704373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.704461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.704610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] resource.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.704690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.704857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.704937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.705100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.705188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.705333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.705413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.705576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.705654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.705816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.705903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.706047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.706126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.706281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.706359 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.706514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.706627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.706769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.706847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.707000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] role.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.707078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.707245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] role.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.707333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.707473 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.707554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.707721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.707800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.707966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.708053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.708209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.708288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.708453 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.708531 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.708696 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.708783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.708926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.709005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.709169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.709246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.709412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.709500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.709640 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.709720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.709884 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.709964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.710126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.710307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.710455 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.710593 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.710814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.710895 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.711065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.711252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.711331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.711488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.711842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.711925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.712008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.712084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.712156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.712225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.712285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.712445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.712535 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.712676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.712757 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.712922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.713001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.713157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.713245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.713400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.713479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.713630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.713710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.713860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.713948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.714090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.714171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.714338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.714417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.714597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.714694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.714912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.715008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.715083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.715154 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.715294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.715384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.715523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.715602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.715755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] token.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.715835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.715990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.716072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.716234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.716314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.716469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.716550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.716749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.716861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.716971 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.717199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.717326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.717455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.717590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.717730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.717857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.717986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.718113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.718241 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.718377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.718504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.718647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.718725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.718797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.718869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.718942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.719080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.719209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.719352 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.719485 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.719611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.719735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.719859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.719991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.720116 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.720240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.720364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.connection = **** {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.720496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.720629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.720754 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.720886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.721010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.721145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.721270 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.721402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.721527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.721659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.721785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.721917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.722041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.722165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.722289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.722421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.722603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.722729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.722872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.723000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.723127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.723332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.723460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.723587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.723715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.723843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.723970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.723970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80222) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 03 15:10:54.724217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.724217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.724217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.724217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.724692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 03 15:10:54.724692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80221) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 03 15:10:54.753798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.754057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.754295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.754625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.754852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.755050 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.762921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.763587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.791708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.791960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.792100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.792235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.847617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.847850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.896729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.897014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.897466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.897701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.966457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.966773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.966959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.967130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.967290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.967449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.967609 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.967772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.967925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.968078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.968231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.968231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.992074 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.992346 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.992484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.992620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.992750 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.992878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.993018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.993149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.993281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.993413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.993413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.993684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.993684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:54.993684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.004920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.005266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.017661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.017871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.018085 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.018224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.018362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.018508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.018691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.018777 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.018903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.019433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.019525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80221) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.019665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.019665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.019665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80222) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 03 15:10:55.037561 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.037778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 15:10:55.038189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.038395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 15:10:55.038696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.038817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.038960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.039077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 15:10:55.039229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.039342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.039484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.039595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 03 15:10:55.039747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.040009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.040260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.040507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.051495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.051771 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.052029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.052306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.052419 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.052649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.052917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.053188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.060453 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.061274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.063799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.064080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.064365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.064627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.064723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.064903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.064990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.065179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.065264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.065455 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.065542 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.065731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.065815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.066043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.066135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.066314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.066397 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.066604 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.066692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.066891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.067161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.067424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.093387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.093844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.095777 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.096050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.096213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.096493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.101425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.101667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.101843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.102090 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.102261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.102261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.109366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.109479 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.109648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.109730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.114816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.114919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.115212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.115318 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.115490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.115574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.115771 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.115853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.116041 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.116130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.116311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.116392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.116580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.116670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.116843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.116923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.135551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.135757 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.135864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.136037 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.140565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.140683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.144260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.144366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.144556 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.144644 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.149525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.149645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.153137 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.153257 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.158920 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.159047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.159220 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.159309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.159502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.159589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.159783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.159866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.160059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.160153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.160333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.160416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.180109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.180387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.180505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.180662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.180750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.180920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.181037 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.181186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.188946 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.189277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.190688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.190964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.191050 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.191272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.193995 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.194274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.194358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.194574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.194663 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.194863 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.194962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.195142 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.195235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.195421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.209061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.209220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.209343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.209495 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.209621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.209771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.209899 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.210098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.210240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.210414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.210496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.210685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.210770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.210958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.229402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.229597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.232691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.232858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.236055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.236281 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.237665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.237976 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.239338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.239619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.239737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.239880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.239963 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.240173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.247324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.247577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.248961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.249485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.252203 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.252525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.252730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.252910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.253098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.253243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.253428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.253607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.262138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.262419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.262695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.262831 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.262979 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.263354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.271348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80222) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.271622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.271898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.272042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80221) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 03 15:10:55.272186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.272328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.272486 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.272579 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.272714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.272797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.272937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.273033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.273208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.273293 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.273480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.273623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.273744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.273830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.274010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.274092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.274283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.274374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.274573 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.274659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.274838 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80222) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.274921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.275100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.275351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80221) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 03 15:10:55.313354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5556ab4cf9c0 pid: 80222 (default app) Jun 03 15:10:55.313824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5556ab4cf9c0 pid: 80221 (default app) Jun 03 15:10:55.316726 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b217d-e1a5-41ba-9c26-644ed15b7bda None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:10:55.317013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b217d-e1a5-41ba-9c26-644ed15b7bda None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:10:55.317290 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b217d-e1a5-41ba-9c26-644ed15b7bda None None] PATH_INFO: `/v3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:10:55.321533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1/1] 10.222.0.93 () {50 vars in 791 bytes} [Fri Jun 3 15:10:55 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:03.723829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda73219-61ac-4131-bae4-4875f4256fb1 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:03.724381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda73219-61ac-4131-bae4-4875f4256fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:03.724475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda73219-61ac-4131-bae4-4875f4256fb1 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:03.729011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1/2] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:03.735071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6748a-e59e-4579-ba79-6b4e6ad49cf6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:03.735621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6748a-e59e-4579-ba79-6b4e6ad49cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:03.735621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6748a-e59e-4579-ba79-6b4e6ad49cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:03.804223 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: INFO dbcounter [None req-78f6748a-e59e-4579-ba79-6b4e6ad49cf6 None None] Registered counter for database keystone Jun 03 15:11:03.826209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG oslo_db.sqlalchemy.engines [None req-78f6748a-e59e-4579-ba79-6b4e6ad49cf6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80222) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 03 15:11:03.833168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writer thread running {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 03 15:11:04.117977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG passlib.handlers.bcrypt [None req-78f6748a-e59e-4579-ba79-6b4e6ad49cf6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80222) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 03 15:11:04.118389 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG passlib.handlers.bcrypt [None req-78f6748a-e59e-4579-ba79-6b4e6ad49cf6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80222) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 03 15:11:04.176434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-78f6748a-e59e-4579-ba79-6b4e6ad49cf6 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:04.178053 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-78f6748a-e59e-4579-ba79-6b4e6ad49cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:04.304694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 2/3] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 571 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 03 15:11:04.471464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda73219-61ac-4131-bae4-4875f4256fb1 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:04.540956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: INFO dbcounter [None req-bda73219-61ac-4131-bae4-4875f4256fb1 None None] Registered counter for database keystone Jun 03 15:11:04.561604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG oslo_db.sqlalchemy.engines [None req-bda73219-61ac-4131-bae4-4875f4256fb1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80221) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 03 15:11:04.568920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writer thread running {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 03 15:11:04.800577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda73219-61ac-4131-bae4-4875f4256fb1 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:04.804959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdf9b2c-e7d9-4fe9-8ee7-c10f5b58175d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:04.805649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf9b2c-e7d9-4fe9-8ee7-c10f5b58175d admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:04.805935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf9b2c-e7d9-4fe9-8ee7-c10f5b58175d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:04.806212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdf9b2c-e7d9-4fe9-8ee7-c10f5b58175d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:04.815133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf9b2c-e7d9-4fe9-8ee7-c10f5b58175d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:04.868458 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2cdf9b2c-e7d9-4fe9-8ee7-c10f5b58175d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 03 15:11:04.868794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdf9b2c-e7d9-4fe9-8ee7-c10f5b58175d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:04.880225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 2/4] 10.222.0.93 () {58 vars in 1324 bytes} [Fri Jun 3 15:11:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:11:04.885652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6748a-e59e-4579-ba79-6b4e6ad49cf6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:04.900481 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6748a-e59e-4579-ba79-6b4e6ad49cf6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:04.905480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c178c-db6b-458b-ad8a-d4f0a4a39d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:04.907143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c178c-db6b-458b-ad8a-d4f0a4a39d94 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:04.907430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c178c-db6b-458b-ad8a-d4f0a4a39d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:04.907707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c178c-db6b-458b-ad8a-d4f0a4a39d94 admin admin] PATH_INFO: `/v3/projects/4b7e9c80374f4695bdf655157e8cb4d3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:04.909233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c178c-db6b-458b-ad8a-d4f0a4a39d94 admin admin] RBAC: Authorizing `identity:get_project(project_id=4b7e9c80374f4695bdf655157e8cb4d3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:04.962425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c56c178c-db6b-458b-ad8a-d4f0a4a39d94 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 03 15:11:04.962792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c178c-db6b-458b-ad8a-d4f0a4a39d94 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:04.967633 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 3/5] 10.222.0.93 () {56 vars in 1207 bytes} [Fri Jun 3 15:11:04 2022] GET /identity/v3/projects/4b7e9c80374f4695bdf655157e8cb4d3 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:07.219921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33f9aa-177b-4efd-a6ba-6a67f6a7b1b6 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:07.220486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33f9aa-177b-4efd-a6ba-6a67f6a7b1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:07.220486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33f9aa-177b-4efd-a6ba-6a67f6a7b1b6 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:07.224628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 3/6] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:07.230771 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba059d3-2be1-4aac-9f2e-e82a764b3eb0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:07.231237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba059d3-2be1-4aac-9f2e-e82a764b3eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:07.231328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba059d3-2be1-4aac-9f2e-e82a764b3eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:07.251656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bba059d3-2be1-4aac-9f2e-e82a764b3eb0 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:07.253224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bba059d3-2be1-4aac-9f2e-e82a764b3eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:07.261276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 4/7] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 03 15:11:07.285195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33f9aa-177b-4efd-a6ba-6a67f6a7b1b6 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:07.299053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33f9aa-177b-4efd-a6ba-6a67f6a7b1b6 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:07.303179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3bb90c-1b03-4904-bd59-b516cedc33e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:07.303811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bb90c-1b03-4904-bd59-b516cedc33e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:07.304091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bb90c-1b03-4904-bd59-b516cedc33e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:07.304365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bb90c-1b03-4904-bd59-b516cedc33e9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:07.312065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3bb90c-1b03-4904-bd59-b516cedc33e9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:07.314402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3bb90c-1b03-4904-bd59-b516cedc33e9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:07.323068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 4/8] 10.222.0.93 () {58 vars in 1324 bytes} [Fri Jun 3 15:11:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:11:07.328961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba059d3-2be1-4aac-9f2e-e82a764b3eb0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:07.338480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba059d3-2be1-4aac-9f2e-e82a764b3eb0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:07.342933 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3000231-90df-48fd-a6f4-130dcd921bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:07.343815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3000231-90df-48fd-a6f4-130dcd921bbd admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:07.344076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3000231-90df-48fd-a6f4-130dcd921bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:07.344349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3000231-90df-48fd-a6f4-130dcd921bbd admin admin] PATH_INFO: `/v3/users/b868f066d8ff4579b72df10988c4ede1` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:07.345740 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3000231-90df-48fd-a6f4-130dcd921bbd admin admin] RBAC: Authorizing `identity:get_user(user_id=b868f066d8ff4579b72df10988c4ede1)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:07.347601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3000231-90df-48fd-a6f4-130dcd921bbd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:07.349179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 5/9] 10.222.0.93 () {56 vars in 1198 bytes} [Fri Jun 3 15:11:07 2022] GET /identity/v3/users/b868f066d8ff4579b72df10988c4ede1 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:09.674891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99911d9-100b-4a51-aca2-480281288a3c None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:09.674891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99911d9-100b-4a51-aca2-480281288a3c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:09.675343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99911d9-100b-4a51-aca2-480281288a3c None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:09.676433 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 5/10] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:09.681876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b486f-7456-4a98-bc4e-101c06bb7641 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:09.682300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b486f-7456-4a98-bc4e-101c06bb7641 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:09.682449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b486f-7456-4a98-bc4e-101c06bb7641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:09.698786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-541b486f-7456-4a98-bc4e-101c06bb7641 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:09.699966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-541b486f-7456-4a98-bc4e-101c06bb7641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:09.721364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce49c4-1fc2-449e-a24f-eb95e86f1de3 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:09.721782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce49c4-1fc2-449e-a24f-eb95e86f1de3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:09.722164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fce49c4-1fc2-449e-a24f-eb95e86f1de3 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:09.723846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 6/11] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:09.729211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384ca59-61bc-4650-a6c3-1d229532659a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:09.729497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384ca59-61bc-4650-a6c3-1d229532659a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:09.729775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384ca59-61bc-4650-a6c3-1d229532659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:09.770618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG passlib.handlers.bcrypt [None req-b384ca59-61bc-4650-a6c3-1d229532659a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80221) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 03 15:11:09.771041 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG passlib.handlers.bcrypt [None req-b384ca59-61bc-4650-a6c3-1d229532659a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80221) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 03 15:11:09.782856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 6/12] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:09.807181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b486f-7456-4a98-bc4e-101c06bb7641 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:09.819363 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-b384ca59-61bc-4650-a6c3-1d229532659a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:09.820955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b384ca59-61bc-4650-a6c3-1d229532659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:09.832386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:09.914462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b486f-7456-4a98-bc4e-101c06bb7641 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:09.921651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 7/13] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:09.944069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384ca59-61bc-4650-a6c3-1d229532659a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:09.948908 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f42d01-a04a-4801-887f-b2d1cc86e41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:09.949712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f42d01-a04a-4801-887f-b2d1cc86e41d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:09.950001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f42d01-a04a-4801-887f-b2d1cc86e41d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:09.950281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f42d01-a04a-4801-887f-b2d1cc86e41d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:09.962602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f42d01-a04a-4801-887f-b2d1cc86e41d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-69f42d01-a04a-4801-887f-b2d1cc86e41d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:09.973906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:09.974944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:09.975778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 7/14] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:11:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:11:09.980853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f42d01-a04a-4801-887f-b2d1cc86e41d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:10.046525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384ca59-61bc-4650-a6c3-1d229532659a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:10.049620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:10.063396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:10.083764 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f42d01-a04a-4801-887f-b2d1cc86e41d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:10.087135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4232a-81fd-49b8-8951-3a3c910e9fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:10.088413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4232a-81fd-49b8-8951-3a3c910e9fde None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:10.088853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4232a-81fd-49b8-8951-3a3c910e9fde None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:10.089138 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4232a-81fd-49b8-8951-3a3c910e9fde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:10.121739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f864b-f21b-408e-b4dd-ebd00a6b1eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:10.122422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f864b-f21b-408e-b4dd-ebd00a6b1eae None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:10.122729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f864b-f21b-408e-b4dd-ebd00a6b1eae None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:10.123009 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f864b-f21b-408e-b4dd-ebd00a6b1eae None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:10.123765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f864b-f21b-408e-b4dd-ebd00a6b1eae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:10.124072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f864b-f21b-408e-b4dd-ebd00a6b1eae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:10.126163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f864b-f21b-408e-b4dd-ebd00a6b1eae None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:10.137682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 8/15] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:10.145163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f864b-f21b-408e-b4dd-ebd00a6b1eae None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:10.161552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4232a-81fd-49b8-8951-3a3c910e9fde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:10.163128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4232a-81fd-49b8-8951-3a3c910e9fde None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:10.246600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f864b-f21b-408e-b4dd-ebd00a6b1eae None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:10.263120 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 8/16] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:10.268806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4232a-81fd-49b8-8951-3a3c910e9fde None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:10.271534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:10.280166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777214b8-f8f6-4bd2-92b5-e0a7305b84a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:10.281079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777214b8-f8f6-4bd2-92b5-e0a7305b84a8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:10.281367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777214b8-f8f6-4bd2-92b5-e0a7305b84a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:10.281646 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777214b8-f8f6-4bd2-92b5-e0a7305b84a8 None admin] PATH_INFO: `/v3/users/b868f066d8ff4579b72df10988c4ede1` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:10.283033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777214b8-f8f6-4bd2-92b5-e0a7305b84a8 None admin] RBAC: Authorizing `identity:get_user(user_id=b868f066d8ff4579b72df10988c4ede1)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:10.284500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777214b8-f8f6-4bd2-92b5-e0a7305b84a8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:10.286149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 9/17] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:10 2022] GET /identity/v3/users/b868f066d8ff4579b72df10988c4ede1 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:10.291723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777214b8-f8f6-4bd2-92b5-e0a7305b84a8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:10.307349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:10.381258 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4232a-81fd-49b8-8951-3a3c910e9fde None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:10.405615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777214b8-f8f6-4bd2-92b5-e0a7305b84a8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:10.420083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea3cf1-8114-4a3d-96ce-e361023aedeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:10.421450 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea3cf1-8114-4a3d-96ce-e361023aedeb None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:10.421952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea3cf1-8114-4a3d-96ce-e361023aedeb None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:10.422386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dea3cf1-8114-4a3d-96ce-e361023aedeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:10.441688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea3cf1-8114-4a3d-96ce-e361023aedeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:10.444678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f97b9-aea3-49fa-90de-ae4ea181e853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:10.444834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dea3cf1-8114-4a3d-96ce-e361023aedeb None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:10.444928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f97b9-aea3-49fa-90de-ae4ea181e853 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:10.444928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f97b9-aea3-49fa-90de-ae4ea181e853 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:10.444928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f97b9-aea3-49fa-90de-ae4ea181e853 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:10.446033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f97b9-aea3-49fa-90de-ae4ea181e853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:10.448698 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f97b9-aea3-49fa-90de-ae4ea181e853 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:10.451860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 10/18] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:10.457945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f97b9-aea3-49fa-90de-ae4ea181e853 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-2dea3cf1-8114-4a3d-96ce-e361023aedeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:11:10.471423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:10.472474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:10.473597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:11:10.473597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:11:10.473597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:11:10.473597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:10.475205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 9/19] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:11:10.480897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea3cf1-8114-4a3d-96ce-e361023aedeb None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:10.564456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:10.574003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:10.600040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f97b9-aea3-49fa-90de-ae4ea181e853 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:10.629048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dea3cf1-8114-4a3d-96ce-e361023aedeb None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:10.639309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55bb8c-d13c-4cd0-816d-fb2f076fa440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:10.640308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55bb8c-d13c-4cd0-816d-fb2f076fa440 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:10.640608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55bb8c-d13c-4cd0-816d-fb2f076fa440 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:10.641410 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55bb8c-d13c-4cd0-816d-fb2f076fa440 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:10.642275 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55bb8c-d13c-4cd0-816d-fb2f076fa440 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.domain.id=default, user.id=b868f066d8ff4579b72df10988c4ede1) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:10.642617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55bb8c-d13c-4cd0-816d-fb2f076fa440 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:10.644195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55bb8c-d13c-4cd0-816d-fb2f076fa440 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:10.652276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 11/20] 10.222.0.93 () {56 vars in 1322 bytes} [Fri Jun 3 15:11:10 2022] GET /identity/v3/role_assignments?user.id=b868f066d8ff4579b72df10988c4ede1&scope.domain.id=default&role.id=d67707782358417ba9f25b2793ca2890 => generated 233 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:10.684431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1542e5f-9fa8-4904-89ba-6198bdc2de10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:10.685141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542e5f-9fa8-4904-89ba-6198bdc2de10 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:10.685431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542e5f-9fa8-4904-89ba-6198bdc2de10 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:10.686336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1542e5f-9fa8-4904-89ba-6198bdc2de10 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:10.687995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1542e5f-9fa8-4904-89ba-6198bdc2de10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:10.688321 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1542e5f-9fa8-4904-89ba-6198bdc2de10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:10.690480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1542e5f-9fa8-4904-89ba-6198bdc2de10 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:10.712257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 10/21] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:13.342746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3af57-8e19-4408-ad23-7b70124a8362 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:13.343259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3af57-8e19-4408-ad23-7b70124a8362 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:13.343331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3af57-8e19-4408-ad23-7b70124a8362 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:13.345490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 12/22] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:13.350855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61a27-bb48-441e-bc5a-a3dc76bfa5ba None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:13.351358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61a27-bb48-441e-bc5a-a3dc76bfa5ba None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:13.351440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61a27-bb48-441e-bc5a-a3dc76bfa5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:13.379735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-a1f61a27-bb48-441e-bc5a-a3dc76bfa5ba None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:13.380969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a1f61a27-bb48-441e-bc5a-a3dc76bfa5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:13.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 11/23] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:13.546330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3af57-8e19-4408-ad23-7b70124a8362 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:13.688863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3af57-8e19-4408-ad23-7b70124a8362 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:13.742092 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e70ca9-18b3-4192-a267-a1b48489aab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:13.743292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e70ca9-18b3-4192-a267-a1b48489aab4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:13.743798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e70ca9-18b3-4192-a267-a1b48489aab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:13.744264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e70ca9-18b3-4192-a267-a1b48489aab4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:13.775892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e70ca9-18b3-4192-a267-a1b48489aab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c2e70ca9-18b3-4192-a267-a1b48489aab4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:13.789212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:13.790790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:13.792395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:13.795905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 13/24] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:11:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:11:13.804314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61a27-bb48-441e-bc5a-a3dc76bfa5ba None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:13.865846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133728a2-0d48-4532-8607-dad6cb49b5f5 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:13.866653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133728a2-0d48-4532-8607-dad6cb49b5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:13.867171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133728a2-0d48-4532-8607-dad6cb49b5f5 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:13.868930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 14/25] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:13.876135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d1af0-b605-454c-9f04-6ac88d3d6c9b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:13.876834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d1af0-b605-454c-9f04-6ac88d3d6c9b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:13.877317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d1af0-b605-454c-9f04-6ac88d3d6c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:13.892336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:13.915002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:13.924667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b69d1af0-b605-454c-9f04-6ac88d3d6c9b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:13.926491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b69d1af0-b605-454c-9f04-6ac88d3d6c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:14.037582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61a27-bb48-441e-bc5a-a3dc76bfa5ba None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:14.060746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 15/26] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:14.062125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d5418-9ebb-45a3-b7a0-1599361a112c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.062987 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d5418-9ebb-45a3-b7a0-1599361a112c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.063282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d5418-9ebb-45a3-b7a0-1599361a112c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.064806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 16/27] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:14.070681 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bbdd0-51db-4fda-9b46-d50c8156771b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.070972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bbdd0-51db-4fda-9b46-d50c8156771b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.071250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bbdd0-51db-4fda-9b46-d50c8156771b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.086042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193ac68-cc87-46f4-915d-8e009bc48b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:14.086755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193ac68-cc87-46f4-915d-8e009bc48b19 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.087041 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193ac68-cc87-46f4-915d-8e009bc48b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.087318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193ac68-cc87-46f4-915d-8e009bc48b19 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.088079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193ac68-cc87-46f4-915d-8e009bc48b19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:14.088386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193ac68-cc87-46f4-915d-8e009bc48b19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:14.089785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193ac68-cc87-46f4-915d-8e009bc48b19 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:14.090350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-140bbdd0-51db-4fda-9b46-d50c8156771b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:14.091604 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-140bbdd0-51db-4fda-9b46-d50c8156771b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:14.107841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 12/28] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:14.108729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193ac68-cc87-46f4-915d-8e009bc48b19 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:14.222235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 17/29] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:14.224645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bbdd0-51db-4fda-9b46-d50c8156771b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:14.315319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193ac68-cc87-46f4-915d-8e009bc48b19 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:14.334952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:14.373969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:14.395209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be8f6b-f74c-47ea-b049-bc9a18dc4c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:14.395900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be8f6b-f74c-47ea-b049-bc9a18dc4c61 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.396187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be8f6b-f74c-47ea-b049-bc9a18dc4c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.396469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be8f6b-f74c-47ea-b049-bc9a18dc4c61 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.397232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be8f6b-f74c-47ea-b049-bc9a18dc4c61 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:14.398657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be8f6b-f74c-47ea-b049-bc9a18dc4c61 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:14.453404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bbdd0-51db-4fda-9b46-d50c8156771b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:14.523286 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e23892-1927-4ba8-adbc-10ec845bb15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:14.524466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e23892-1927-4ba8-adbc-10ec845bb15c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.524897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e23892-1927-4ba8-adbc-10ec845bb15c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.525317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e23892-1927-4ba8-adbc-10ec845bb15c None admin] PATH_INFO: `/v3/users/b868f066d8ff4579b72df10988c4ede1` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.526701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e23892-1927-4ba8-adbc-10ec845bb15c None admin] RBAC: Authorizing `identity:get_user(user_id=b868f066d8ff4579b72df10988c4ede1)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:14.528336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e23892-1927-4ba8-adbc-10ec845bb15c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:14.530811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 18/30] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:14 2022] GET /identity/v3/users/b868f066d8ff4579b72df10988c4ede1 => generated 247 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:14.536524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e23892-1927-4ba8-adbc-10ec845bb15c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:14.590543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 13/31] 10.222.0.93 () {60 vars in 1131 bytes} [Fri Jun 3 15:11:14 2022] POST /identity/v3/roles => generated 219 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:14.592857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecd3f9-d3e2-4eef-91a5-30ad03b71063 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.593319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecd3f9-d3e2-4eef-91a5-30ad03b71063 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.594292 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecd3f9-d3e2-4eef-91a5-30ad03b71063 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.596250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 14/32] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:14.597551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecd3f9-d3e2-4eef-91a5-30ad03b71063 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:14.669565 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e23892-1927-4ba8-adbc-10ec845bb15c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:14.707777 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:14.714245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f76e1-29cf-4a8a-839f-48d40dc252aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:14.715517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f76e1-29cf-4a8a-839f-48d40dc252aa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.715999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f76e1-29cf-4a8a-839f-48d40dc252aa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.716459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f76e1-29cf-4a8a-839f-48d40dc252aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.730009 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f76e1-29cf-4a8a-839f-48d40dc252aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:14.732384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061f76e1-29cf-4a8a-839f-48d40dc252aa None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:14.736737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecd3f9-d3e2-4eef-91a5-30ad03b71063 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-061f76e1-29cf-4a8a-839f-48d40dc252aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:11:14.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:14.755376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:14.756167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:11:14.756167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:11:14.756167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:11:14.756167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:14.764298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 19/33] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:14.766869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d75664-ef80-4e1d-ae36-42b5b4fa1d57 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.767369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d75664-ef80-4e1d-ae36-42b5b4fa1d57 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.767821 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d75664-ef80-4e1d-ae36-42b5b4fa1d57 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.769610 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 20/34] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:14.770980 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f6360-90c5-451a-8093-8f0855db582a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.771478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f6360-90c5-451a-8093-8f0855db582a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.771934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f6360-90c5-451a-8093-8f0855db582a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.778834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19718083-26fd-47be-b3b3-3057df07c588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:14.779986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19718083-26fd-47be-b3b3-3057df07c588 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.780496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19718083-26fd-47be-b3b3-3057df07c588 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.780987 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19718083-26fd-47be-b3b3-3057df07c588 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.783808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19718083-26fd-47be-b3b3-3057df07c588 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:14.785922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19718083-26fd-47be-b3b3-3057df07c588 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:14.787986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 15/35] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:14.789074 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19718083-26fd-47be-b3b3-3057df07c588 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:14.791910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c01f6360-90c5-451a-8093-8f0855db582a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:14.793312 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c01f6360-90c5-451a-8093-8f0855db582a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:14.832895 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:14.888770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 21/36] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:14.890855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc580c0-086c-46b3-b107-8e124d7af897 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.891290 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc580c0-086c-46b3-b107-8e124d7af897 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.891736 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc580c0-086c-46b3-b107-8e124d7af897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.917092 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8cc580c0-086c-46b3-b107-8e124d7af897 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:14.918287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8cc580c0-086c-46b3-b107-8e124d7af897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:14.920402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19718083-26fd-47be-b3b3-3057df07c588 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:14.960024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1eb0d5-1fa5-4f9a-91b8-68be8338b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:14.960939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eb0d5-1fa5-4f9a-91b8-68be8338b3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:14.961374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eb0d5-1fa5-4f9a-91b8-68be8338b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:14.961800 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eb0d5-1fa5-4f9a-91b8-68be8338b3f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:14.962735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1eb0d5-1fa5-4f9a-91b8-68be8338b3f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:14.963193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1eb0d5-1fa5-4f9a-91b8-68be8338b3f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:14.964612 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1eb0d5-1fa5-4f9a-91b8-68be8338b3f9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:14.980441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 16/37] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:14.981629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1eb0d5-1fa5-4f9a-91b8-68be8338b3f9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:14.995412 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 22/38] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:14.999470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc580c0-086c-46b3-b107-8e124d7af897 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:15.059637 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:15.115834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:15.124712 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1eb0d5-1fa5-4f9a-91b8-68be8338b3f9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:15.158109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc580c0-086c-46b3-b107-8e124d7af897 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:15.174885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ac09a-efd0-4a33-a2bd-b7255c43b7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:15.175766 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac09a-efd0-4a33-a2bd-b7255c43b7f9 None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:15.176052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac09a-efd0-4a33-a2bd-b7255c43b7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:15.176332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac09a-efd0-4a33-a2bd-b7255c43b7f9 None admin] PATH_INFO: `/v3/domains/default/users/b868f066d8ff4579b72df10988c4ede1/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:15.178788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ac09a-efd0-4a33-a2bd-b7255c43b7f9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b868f066d8ff4579b72df10988c4ede1, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:15.180277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ac09a-efd0-4a33-a2bd-b7255c43b7f9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:15.198754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 17/39] 10.222.0.93 () {58 vars in 1361 bytes} [Fri Jun 3 15:11:14 2022] PUT /identity/v3/domains/default/users/b868f066d8ff4579b72df10988c4ede1/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:11:15.199576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ac09a-efd0-4a33-a2bd-b7255c43b7f9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:15.214291 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531560e-5c53-4282-8772-1ef9ac667c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:15.216645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531560e-5c53-4282-8772-1ef9ac667c1a None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:15.217173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531560e-5c53-4282-8772-1ef9ac667c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:15.217675 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531560e-5c53-4282-8772-1ef9ac667c1a None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:15.219250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531560e-5c53-4282-8772-1ef9ac667c1a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:15.221857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531560e-5c53-4282-8772-1ef9ac667c1a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:15.347930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ac09a-efd0-4a33-a2bd-b7255c43b7f9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:15.361438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 23/40] 10.222.0.93 () {60 vars in 1131 bytes} [Fri Jun 3 15:11:14 2022] POST /identity/v3/roles => generated 225 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:15.362467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531560e-5c53-4282-8772-1ef9ac667c1a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:15.406946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409e6644-ff4e-4d1a-82aa-adcf8ea89816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:15.409516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e6644-ff4e-4d1a-82aa-adcf8ea89816 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:15.410267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e6644-ff4e-4d1a-82aa-adcf8ea89816 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:15.412141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409e6644-ff4e-4d1a-82aa-adcf8ea89816 None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:15.413564 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e6644-ff4e-4d1a-82aa-adcf8ea89816 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:15.416590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409e6644-ff4e-4d1a-82aa-adcf8ea89816 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:15.454229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 18/41] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:11:15 2022] POST /identity/v3/projects => generated 301 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:15.455739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e2f94-c2c3-419c-acb4-d7a954235f1d None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:15.456035 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e2f94-c2c3-419c-acb4-d7a954235f1d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:15.456318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e2f94-c2c3-419c-acb4-d7a954235f1d None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:15.457971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 19/42] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:15.464106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124ae49-6fe1-403d-a3bf-c30c9f2c7d89 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:15.464459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124ae49-6fe1-403d-a3bf-c30c9f2c7d89 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:15.464747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124ae49-6fe1-403d-a3bf-c30c9f2c7d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:15.501065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2124ae49-6fe1-403d-a3bf-c30c9f2c7d89 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:15.505876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2124ae49-6fe1-403d-a3bf-c30c9f2c7d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:15.512405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:15.539871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531560e-5c53-4282-8772-1ef9ac667c1a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:15.546886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:15.621478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d4dc3-b0b2-4adf-b6ff-cf38c4a714e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:15.622138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d4dc3-b0b2-4adf-b6ff-cf38c4a714e6 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:15.622452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d4dc3-b0b2-4adf-b6ff-cf38c4a714e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:15.622992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d4dc3-b0b2-4adf-b6ff-cf38c4a714e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:15.624035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d4dc3-b0b2-4adf-b6ff-cf38c4a714e6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:15.625790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d4dc3-b0b2-4adf-b6ff-cf38c4a714e6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:15.628920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 20/43] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:15.647874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 24/44] 10.222.0.93 () {60 vars in 1131 bytes} [Fri Jun 3 15:11:15 2022] POST /identity/v3/roles => generated 223 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:15.659364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124ae49-6fe1-403d-a3bf-c30c9f2c7d89 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:15.796510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124ae49-6fe1-403d-a3bf-c30c9f2c7d89 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:15.839859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb15f7c-9960-4b18-89c6-7607fa009666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:15.840704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb15f7c-9960-4b18-89c6-7607fa009666 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:15.840991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb15f7c-9960-4b18-89c6-7607fa009666 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:15.841282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb15f7c-9960-4b18-89c6-7607fa009666 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:15.842759 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb15f7c-9960-4b18-89c6-7607fa009666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:15.844288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb15f7c-9960-4b18-89c6-7607fa009666 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:15.846366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 21/45] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:15.852032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d4dc3-b0b2-4adf-b6ff-cf38c4a714e6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:15.962569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d4dc3-b0b2-4adf-b6ff-cf38c4a714e6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:15.996425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835dc805-2750-4ca8-8d29-4caf8e8b6c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:15.997107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dc805-2750-4ca8-8d29-4caf8e8b6c89 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:15.997392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dc805-2750-4ca8-8d29-4caf8e8b6c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:15.997657 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dc805-2750-4ca8-8d29-4caf8e8b6c89 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:15.998386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835dc805-2750-4ca8-8d29-4caf8e8b6c89 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:16.000430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835dc805-2750-4ca8-8d29-4caf8e8b6c89 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:16.042321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 25/46] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:11:15 2022] POST /identity/v3/projects => generated 312 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:17.592775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b296f7b-6a15-4856-8ff2-8e7c075c6440 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:17.593330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b296f7b-6a15-4856-8ff2-8e7c075c6440 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:17.593416 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b296f7b-6a15-4856-8ff2-8e7c075c6440 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:17.595137 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 22/47] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:17.602427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef814d7-6ccb-40ef-96eb-ed981ef100df None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:17.602939 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef814d7-6ccb-40ef-96eb-ed981ef100df None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:17.603025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef814d7-6ccb-40ef-96eb-ed981ef100df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:17.620619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-fef814d7-6ccb-40ef-96eb-ed981ef100df None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:17.622167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fef814d7-6ccb-40ef-96eb-ed981ef100df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:17.635797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:17.705319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 26/48] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:17.728061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b296f7b-6a15-4856-8ff2-8e7c075c6440 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:17.737584 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:17.836947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b296f7b-6a15-4856-8ff2-8e7c075c6440 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:17.871014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b1a5c-ebc1-40a8-a764-5d931234a279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:17.871717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1a5c-ebc1-40a8-a764-5d931234a279 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:17.872001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1a5c-ebc1-40a8-a764-5d931234a279 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:17.872420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1a5c-ebc1-40a8-a764-5d931234a279 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:17.883970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b1a5c-ebc1-40a8-a764-5d931234a279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-6f9b1a5c-ebc1-40a8-a764-5d931234a279 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:17.892877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:17.893749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:17.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 23/49] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:11:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:17.899760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef814d7-6ccb-40ef-96eb-ed981ef100df None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:18.010927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef814d7-6ccb-40ef-96eb-ed981ef100df None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:18.067682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80f171-a669-4286-9a08-eb9d22c3d4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:18.068669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80f171-a669-4286-9a08-eb9d22c3d4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:18.069098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80f171-a669-4286-9a08-eb9d22c3d4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:18.069511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80f171-a669-4286-9a08-eb9d22c3d4b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:18.070720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80f171-a669-4286-9a08-eb9d22c3d4b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:18.071203 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80f171-a669-4286-9a08-eb9d22c3d4b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:18.073518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80f171-a669-4286-9a08-eb9d22c3d4b9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:18.090332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 27/50] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:18.091512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b1a5c-ebc1-40a8-a764-5d931234a279 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:18.163746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:18.211163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b1a5c-ebc1-40a8-a764-5d931234a279 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:18.254271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738f160-e0f5-4085-8b1b-3ba296dbf6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:18.255201 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738f160-e0f5-4085-8b1b-3ba296dbf6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:18.255522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738f160-e0f5-4085-8b1b-3ba296dbf6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:18.255761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738f160-e0f5-4085-8b1b-3ba296dbf6b9 None admin] PATH_INFO: `/v3/users/b868f066d8ff4579b72df10988c4ede1` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:18.257031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738f160-e0f5-4085-8b1b-3ba296dbf6b9 None admin] RBAC: Authorizing `identity:get_user(user_id=b868f066d8ff4579b72df10988c4ede1)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:18.259572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738f160-e0f5-4085-8b1b-3ba296dbf6b9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:18.262820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 24/51] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:18 2022] GET /identity/v3/users/b868f066d8ff4579b72df10988c4ede1 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:18.271117 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80f171-a669-4286-9a08-eb9d22c3d4b9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:18.289343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:18.379358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e12e1-3bf8-483f-82f9-32d690be1052 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:18.379996 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e12e1-3bf8-483f-82f9-32d690be1052 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:18.380150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e12e1-3bf8-483f-82f9-32d690be1052 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:18.381478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 25/52] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:11:18.387614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744b579-55fa-43c5-a51d-09d6c888bb52 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:18.387904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744b579-55fa-43c5-a51d-09d6c888bb52 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:18.388331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744b579-55fa-43c5-a51d-09d6c888bb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:18.416587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-5744b579-55fa-43c5-a51d-09d6c888bb52 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:18.418082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5744b579-55fa-43c5-a51d-09d6c888bb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:18.444968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80f171-a669-4286-9a08-eb9d22c3d4b9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:18.488182 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e328c4d-426c-45c5-9bd0-a310e3656e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:18.489031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e328c4d-426c-45c5-9bd0-a310e3656e78 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:18.490059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e328c4d-426c-45c5-9bd0-a310e3656e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:18.491617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e328c4d-426c-45c5-9bd0-a310e3656e78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:18.493146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e328c4d-426c-45c5-9bd0-a310e3656e78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:18.494793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e328c4d-426c-45c5-9bd0-a310e3656e78 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:18.496677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 28/53] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:18.503915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e328c4d-426c-45c5-9bd0-a310e3656e78 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:18.540483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 26/54] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:18.576621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744b579-55fa-43c5-a51d-09d6c888bb52 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:18.612139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:18.641946 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e328c4d-426c-45c5-9bd0-a310e3656e78 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:18.665402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:18.687254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214c1f8-2ed3-4281-a74e-25fc9ea2ec25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:18.687921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214c1f8-2ed3-4281-a74e-25fc9ea2ec25 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:18.688222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214c1f8-2ed3-4281-a74e-25fc9ea2ec25 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:18.688473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214c1f8-2ed3-4281-a74e-25fc9ea2ec25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:18.689318 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214c1f8-2ed3-4281-a74e-25fc9ea2ec25 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.domain.id=default, user.id=b868f066d8ff4579b72df10988c4ede1) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:18.689631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214c1f8-2ed3-4281-a74e-25fc9ea2ec25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:18.691147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214c1f8-2ed3-4281-a74e-25fc9ea2ec25 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:18.697492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 29/55] 10.222.0.93 () {56 vars in 1322 bytes} [Fri Jun 3 15:11:18 2022] GET /identity/v3/role_assignments?user.id=b868f066d8ff4579b72df10988c4ede1&scope.domain.id=default&role.id=d67707782358417ba9f25b2793ca2890 => generated 530 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:18.722263 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744b579-55fa-43c5-a51d-09d6c888bb52 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:18.778833 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6369e-1450-40b4-b45d-f1d4725ef457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:18.780230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6369e-1450-40b4-b45d-f1d4725ef457 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:18.780722 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6369e-1450-40b4-b45d-f1d4725ef457 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:18.781808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6369e-1450-40b4-b45d-f1d4725ef457 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:18.784083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6369e-1450-40b4-b45d-f1d4725ef457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:18.786993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6369e-1450-40b4-b45d-f1d4725ef457 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:18.790239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 27/56] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:18.796994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214c1f8-2ed3-4281-a74e-25fc9ea2ec25 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:18.927812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214c1f8-2ed3-4281-a74e-25fc9ea2ec25 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:18.970516 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e00bf5-e541-48de-817e-228b374a057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:18.971214 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e00bf5-e541-48de-817e-228b374a057c None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:18.971502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e00bf5-e541-48de-817e-228b374a057c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:18.971784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e00bf5-e541-48de-817e-228b374a057c None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:18.972508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e00bf5-e541-48de-817e-228b374a057c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:18.974001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e00bf5-e541-48de-817e-228b374a057c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:18.985593 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:19.012379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 30/57] 10.222.0.93 () {60 vars in 1140 bytes} [Fri Jun 3 15:11:18 2022] POST /identity/v3/projects => generated 298 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:21.366357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd4f50-1879-4665-ab69-a38942f8a212 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:21.367132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd4f50-1879-4665-ab69-a38942f8a212 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:21.367132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd4f50-1879-4665-ab69-a38942f8a212 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:21.368322 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 28/58] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:21.374299 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019b27-2516-457f-8aa2-ee08ebda3880 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:21.374933 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019b27-2516-457f-8aa2-ee08ebda3880 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:21.375118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45019b27-2516-457f-8aa2-ee08ebda3880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:21.392636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-45019b27-2516-457f-8aa2-ee08ebda3880 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:21.393862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-45019b27-2516-457f-8aa2-ee08ebda3880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:21.474723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 31/59] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:21.499720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd4f50-1879-4665-ab69-a38942f8a212 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:21.614668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd4f50-1879-4665-ab69-a38942f8a212 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:21.646761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:21.653803 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9496d-5066-4169-a2af-dbcfe0e1b05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:21.655068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9496d-5066-4169-a2af-dbcfe0e1b05c None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:21.655499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9496d-5066-4169-a2af-dbcfe0e1b05c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:21.656526 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9496d-5066-4169-a2af-dbcfe0e1b05c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:21.658319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9496d-5066-4169-a2af-dbcfe0e1b05c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:21.660921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9496d-5066-4169-a2af-dbcfe0e1b05c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:21.663230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 29/60] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:21.671924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45019b27-2516-457f-8aa2-ee08ebda3880 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:21.800923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45019b27-2516-457f-8aa2-ee08ebda3880 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:21.834371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:21.850263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e084f-79b6-4f17-a950-61252f62b4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:21.850973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e084f-79b6-4f17-a950-61252f62b4ed None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:21.851261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e084f-79b6-4f17-a950-61252f62b4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:21.851539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e084f-79b6-4f17-a950-61252f62b4ed None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:21.852261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e084f-79b6-4f17-a950-61252f62b4ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:21.853817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e084f-79b6-4f17-a950-61252f62b4ed None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:21.889145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 32/61] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:11:21 2022] POST /identity/v3/users => generated 275 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:25.963733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece604e6-e70a-4290-8093-92681ce626e6 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:25.964209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece604e6-e70a-4290-8093-92681ce626e6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:25.964209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece604e6-e70a-4290-8093-92681ce626e6 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:25.965589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 30/62] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:11:25.978315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5ebde-8165-4aaa-b8ab-a90c21fe0ed7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:25.979292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5ebde-8165-4aaa-b8ab-a90c21fe0ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:25.979779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5ebde-8165-4aaa-b8ab-a90c21fe0ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:26.007624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0fe5ebde-8165-4aaa-b8ab-a90c21fe0ed7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:26.009320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0fe5ebde-8165-4aaa-b8ab-a90c21fe0ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:26.111982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 33/63] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:26.171190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece604e6-e70a-4290-8093-92681ce626e6 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:26.326648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece604e6-e70a-4290-8093-92681ce626e6 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:26.369683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0d52e-3fe6-4b40-abf7-aafe8874d4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:26.370495 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0d52e-3fe6-4b40-abf7-aafe8874d4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:26.370805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0d52e-3fe6-4b40-abf7-aafe8874d4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:26.371080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0d52e-3fe6-4b40-abf7-aafe8874d4e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:26.396151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0d52e-3fe6-4b40-abf7-aafe8874d4e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-18a0d52e-3fe6-4b40-abf7-aafe8874d4e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:26.406451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:26.407411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:26.408183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 31/64] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:26.419675 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5ebde-8165-4aaa-b8ab-a90c21fe0ed7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:26.444756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897876a2-7411-4509-87ba-f6a1fdf32162 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:26.445405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897876a2-7411-4509-87ba-f6a1fdf32162 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:26.445862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897876a2-7411-4509-87ba-f6a1fdf32162 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:26.447569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 32/65] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:11:26.453282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcaa8a-ad60-46e8-a597-cd757921b757 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:26.453871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcaa8a-ad60-46e8-a597-cd757921b757 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:26.454464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcaa8a-ad60-46e8-a597-cd757921b757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:26.480079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-a0fcaa8a-ad60-46e8-a597-cd757921b757 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:26.481915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a0fcaa8a-ad60-46e8-a597-cd757921b757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:26.585468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:26.597286 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5ebde-8165-4aaa-b8ab-a90c21fe0ed7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:26.598086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 33/66] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:26.599824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f26e51-cc47-4753-9d38-d62a0f41884f None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:26.600294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f26e51-cc47-4753-9d38-d62a0f41884f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:26.600745 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f26e51-cc47-4753-9d38-d62a0f41884f None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:26.602272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 34/67] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:26.608743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:26.610379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382697d-d9ba-461b-8d1d-a2b364f5fe4c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:26.611446 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382697d-d9ba-461b-8d1d-a2b364f5fe4c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:26.611940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382697d-d9ba-461b-8d1d-a2b364f5fe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:26.633472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-5382697d-d9ba-461b-8d1d-a2b364f5fe4c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:26.635155 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5382697d-d9ba-461b-8d1d-a2b364f5fe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:26.659085 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbd1a1-7b3a-4333-8083-f087d2b19ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:26.659447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbd1a1-7b3a-4333-8083-f087d2b19ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:26.659655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbd1a1-7b3a-4333-8083-f087d2b19ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:26.659938 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbd1a1-7b3a-4333-8083-f087d2b19ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:26.660713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbd1a1-7b3a-4333-8083-f087d2b19ed9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:26.661024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbd1a1-7b3a-4333-8083-f087d2b19ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:26.662477 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbd1a1-7b3a-4333-8083-f087d2b19ed9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:26.680459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 34/68] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:26.680459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbd1a1-7b3a-4333-8083-f087d2b19ed9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:26.716190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 35/69] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:26.720924 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3343e2-3f6a-40d8-9d63-fd46c1724dd5 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:26.722712 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3343e2-3f6a-40d8-9d63-fd46c1724dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:26.723229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3343e2-3f6a-40d8-9d63-fd46c1724dd5 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:26.724976 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 36/70] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:26.726033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3343e2-3f6a-40d8-9d63-fd46c1724dd5 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:26.812793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbd1a1-7b3a-4333-8083-f087d2b19ed9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:26.850488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3343e2-3f6a-40d8-9d63-fd46c1724dd5 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:26.853714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577c13a-42d7-4081-8310-3c3eff779ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:26.854541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577c13a-42d7-4081-8310-3c3eff779ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:26.854859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577c13a-42d7-4081-8310-3c3eff779ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:26.855140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577c13a-42d7-4081-8310-3c3eff779ea8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:26.866415 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3577c13a-42d7-4081-8310-3c3eff779ea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:26.874082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-3577c13a-42d7-4081-8310-3c3eff779ea8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:26.874678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:26.875512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:26.876297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 35/71] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:11:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:26.877609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20d1e7-abd4-4671-9aac-71c6e4dd87ae None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:26.877609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20d1e7-abd4-4671-9aac-71c6e4dd87ae None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:26.877754 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20d1e7-abd4-4671-9aac-71c6e4dd87ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:26.894412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e13a4d-8793-43d0-82c6-9cb04b171f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:26.895308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e13a4d-8793-43d0-82c6-9cb04b171f8b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:26.895597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e13a4d-8793-43d0-82c6-9cb04b171f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:26.895882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e13a4d-8793-43d0-82c6-9cb04b171f8b None admin] PATH_INFO: `/v3/users/8d32be869c6c48259625a2926009e24a` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:26.896679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-dc20d1e7-abd4-4671-9aac-71c6e4dd87ae None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:26.897902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-dc20d1e7-abd4-4671-9aac-71c6e4dd87ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:26.913406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e13a4d-8793-43d0-82c6-9cb04b171f8b None admin] RBAC: Authorizing `identity:get_user(user_id=8d32be869c6c48259625a2926009e24a)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:26.914905 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e13a4d-8793-43d0-82c6-9cb04b171f8b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:26.916901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 37/72] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:26 2022] GET /identity/v3/users/8d32be869c6c48259625a2926009e24a => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:26.922943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e13a4d-8793-43d0-82c6-9cb04b171f8b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:26.925335 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:26.983462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 36/73] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:26.985521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20d1e7-abd4-4671-9aac-71c6e4dd87ae None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:27.056860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e13a4d-8793-43d0-82c6-9cb04b171f8b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:27.104078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7dfb0-4f92-4ac7-b4c3-36a7399449b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:27.104889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7dfb0-4f92-4ac7-b4c3-36a7399449b2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:27.105195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7dfb0-4f92-4ac7-b4c3-36a7399449b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:27.105467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7dfb0-4f92-4ac7-b4c3-36a7399449b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:27.110106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20d1e7-abd4-4671-9aac-71c6e4dd87ae None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:27.116393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7dfb0-4f92-4ac7-b4c3-36a7399449b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-6fd7dfb0-4f92-4ac7-b4c3-36a7399449b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:27.125231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:27.126206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:27.127043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:27.130690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 38/74] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:27.132188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7dfb0-4f92-4ac7-b4c3-36a7399449b2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:27.149756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecccd98-1f61-4a12-b65d-6bded980e5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:27.150436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecccd98-1f61-4a12-b65d-6bded980e5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:27.150783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecccd98-1f61-4a12-b65d-6bded980e5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:27.151066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecccd98-1f61-4a12-b65d-6bded980e5a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:27.151832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecccd98-1f61-4a12-b65d-6bded980e5a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:27.152382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecccd98-1f61-4a12-b65d-6bded980e5a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:27.153548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecccd98-1f61-4a12-b65d-6bded980e5a4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:27.162092 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 37/75] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:11:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:27.163070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecccd98-1f61-4a12-b65d-6bded980e5a4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:27.180138 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:27.191802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:27.244448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7dfb0-4f92-4ac7-b4c3-36a7399449b2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:27.276040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecccd98-1f61-4a12-b65d-6bded980e5a4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:27.284136 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362651ae-7824-4267-9ffe-a00cc13bd75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:27.284843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362651ae-7824-4267-9ffe-a00cc13bd75b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:27.285132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362651ae-7824-4267-9ffe-a00cc13bd75b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:27.285411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362651ae-7824-4267-9ffe-a00cc13bd75b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:27.315720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c18fc4-4289-471d-95e8-521fd39cfa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:27.316835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18fc4-4289-471d-95e8-521fd39cfa8a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:27.317247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18fc4-4289-471d-95e8-521fd39cfa8a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:27.317659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c18fc4-4289-471d-95e8-521fd39cfa8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:27.331300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c18fc4-4289-471d-95e8-521fd39cfa8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-42c18fc4-4289-471d-95e8-521fd39cfa8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:27.340804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:27.342014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:27.343225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 38/76] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:11:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:27.344116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d267321-3c59-45cb-931a-8d1f3a63f606 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:27.344116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d267321-3c59-45cb-931a-8d1f3a63f606 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:27.344440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d267321-3c59-45cb-931a-8d1f3a63f606 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:27.345934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 39/77] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:27.347276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d267321-3c59-45cb-931a-8d1f3a63f606 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:27.369076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362651ae-7824-4267-9ffe-a00cc13bd75b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:27.370497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362651ae-7824-4267-9ffe-a00cc13bd75b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:27.421428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:27.452019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:27.477900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d267321-3c59-45cb-931a-8d1f3a63f606 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:27.498635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 39/78] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:27.500707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362651ae-7824-4267-9ffe-a00cc13bd75b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:27.528096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff81a17-5aec-43da-876b-8d8dcef65f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:27.529075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff81a17-5aec-43da-876b-8d8dcef65f29 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:27.529370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff81a17-5aec-43da-876b-8d8dcef65f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:27.529647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff81a17-5aec-43da-876b-8d8dcef65f29 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:27.530411 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff81a17-5aec-43da-876b-8d8dcef65f29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:27.530892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff81a17-5aec-43da-876b-8d8dcef65f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:27.532954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff81a17-5aec-43da-876b-8d8dcef65f29 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:27.547754 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 40/79] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:27.549116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff81a17-5aec-43da-876b-8d8dcef65f29 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:27.638517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362651ae-7824-4267-9ffe-a00cc13bd75b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:27.682514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff81a17-5aec-43da-876b-8d8dcef65f29 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:27.691243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eb1cb-324f-4ed7-97f4-32635d378e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:27.692344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eb1cb-324f-4ed7-97f4-32635d378e25 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:27.692628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eb1cb-324f-4ed7-97f4-32635d378e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:27.692904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eb1cb-324f-4ed7-97f4-32635d378e25 None admin] PATH_INFO: `/v3/users/8d32be869c6c48259625a2926009e24a` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:27.694096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19eb1cb-324f-4ed7-97f4-32635d378e25 None admin] RBAC: Authorizing `identity:get_user(user_id=8d32be869c6c48259625a2926009e24a)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:27.695582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19eb1cb-324f-4ed7-97f4-32635d378e25 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:27.697205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 40/80] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:27 2022] GET /identity/v3/users/8d32be869c6c48259625a2926009e24a => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:27.698424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a09d2-1f7c-4b99-8118-38dac204f772 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:27.698730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a09d2-1f7c-4b99-8118-38dac204f772 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:27.699010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a09d2-1f7c-4b99-8118-38dac204f772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:27.709538 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:27.716321 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-757a09d2-1f7c-4b99-8118-38dac204f772 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:27.717543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-757a09d2-1f7c-4b99-8118-38dac204f772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:27.721506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce2115-8d7b-41b1-9cda-62fc98779615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:27.722180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2115-8d7b-41b1-9cda-62fc98779615 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:27.722468 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2115-8d7b-41b1-9cda-62fc98779615 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:27.722778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2115-8d7b-41b1-9cda-62fc98779615 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:27.723533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce2115-8d7b-41b1-9cda-62fc98779615 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:27.723841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce2115-8d7b-41b1-9cda-62fc98779615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:27.725226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce2115-8d7b-41b1-9cda-62fc98779615 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:27.735067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 41/81] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:27.735655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce2115-8d7b-41b1-9cda-62fc98779615 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:27.744247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:27.790420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 41/82] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:27.791574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a09d2-1f7c-4b99-8118-38dac204f772 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:27.835737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce2115-8d7b-41b1-9cda-62fc98779615 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:27.869533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b04f6-f2b3-49de-9073-8a0c9ef9e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:27.870391 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b04f6-f2b3-49de-9073-8a0c9ef9e0fb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:27.870706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b04f6-f2b3-49de-9073-8a0c9ef9e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:27.870984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b04f6-f2b3-49de-9073-8a0c9ef9e0fb None admin] PATH_INFO: `/v3/projects/836a895c1fd443928a00b4d27d10a8f9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:27.872177 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b04f6-f2b3-49de-9073-8a0c9ef9e0fb None admin] RBAC: Authorizing `identity:get_project(project_id=836a895c1fd443928a00b4d27d10a8f9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:27.873660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b04f6-f2b3-49de-9073-8a0c9ef9e0fb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:27.875441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 42/83] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:27 2022] GET /identity/v3/projects/836a895c1fd443928a00b4d27d10a8f9 => generated 312 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:27.876213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b04f6-f2b3-49de-9073-8a0c9ef9e0fb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:27.903714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a09d2-1f7c-4b99-8118-38dac204f772 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:27.943795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8accfe-ea92-4e19-82d3-4f2152370153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:27.944702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8accfe-ea92-4e19-82d3-4f2152370153 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:27.944989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8accfe-ea92-4e19-82d3-4f2152370153 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:27.945268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8accfe-ea92-4e19-82d3-4f2152370153 None admin] PATH_INFO: `/v3/users/8d32be869c6c48259625a2926009e24a` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:27.946473 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8accfe-ea92-4e19-82d3-4f2152370153 None admin] RBAC: Authorizing `identity:get_user(user_id=8d32be869c6c48259625a2926009e24a)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:27.947973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8accfe-ea92-4e19-82d3-4f2152370153 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:27.949616 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 42/84] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:27 2022] GET /identity/v3/users/8d32be869c6c48259625a2926009e24a => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:27.950465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8accfe-ea92-4e19-82d3-4f2152370153 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:27.954056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:27.961291 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:27.987103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b04f6-f2b3-49de-9073-8a0c9ef9e0fb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:28.019569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319332b-219f-4ce7-90ff-bb670bddcb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:28.020210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319332b-219f-4ce7-90ff-bb670bddcb99 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:28.020491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319332b-219f-4ce7-90ff-bb670bddcb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:28.020767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319332b-219f-4ce7-90ff-bb670bddcb99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:28.052037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8accfe-ea92-4e19-82d3-4f2152370153 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:28.085090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf655828-0920-4946-8dc5-5c31b04c0d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:28.085946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf655828-0920-4946-8dc5-5c31b04c0d91 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:28.086222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf655828-0920-4946-8dc5-5c31b04c0d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:28.086497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf655828-0920-4946-8dc5-5c31b04c0d91 None admin] PATH_INFO: `/v3/users/b868f066d8ff4579b72df10988c4ede1` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:28.087720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf655828-0920-4946-8dc5-5c31b04c0d91 None admin] RBAC: Authorizing `identity:get_user(user_id=b868f066d8ff4579b72df10988c4ede1)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:28.089062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319332b-219f-4ce7-90ff-bb670bddcb99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:28.089178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf655828-0920-4946-8dc5-5c31b04c0d91 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:28.090739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319332b-219f-4ce7-90ff-bb670bddcb99 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:28.096038 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 43/85] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:27 2022] GET /identity/v3/users/b868f066d8ff4579b72df10988c4ede1 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:28.096038 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf655828-0920-4946-8dc5-5c31b04c0d91 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:28.162451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:28.169813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:28.193669 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf655828-0920-4946-8dc5-5c31b04c0d91 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:28.196346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 43/86] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:28.197168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319332b-219f-4ce7-90ff-bb670bddcb99 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:28.229737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819006d4-a1f4-4e56-8701-fa4a4684335e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:28.230524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819006d4-a1f4-4e56-8701-fa4a4684335e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:28.230830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819006d4-a1f4-4e56-8701-fa4a4684335e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:28.231177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819006d4-a1f4-4e56-8701-fa4a4684335e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:28.232621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819006d4-a1f4-4e56-8701-fa4a4684335e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:28.234145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819006d4-a1f4-4e56-8701-fa4a4684335e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:28.235843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 44/87] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:28.236621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819006d4-a1f4-4e56-8701-fa4a4684335e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:28.302664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319332b-219f-4ce7-90ff-bb670bddcb99 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:28.336187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3d15d-44cc-4832-8d63-6e97a2edfaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:28.336841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3d15d-44cc-4832-8d63-6e97a2edfaac None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:28.337130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3d15d-44cc-4832-8d63-6e97a2edfaac None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:28.337420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3d15d-44cc-4832-8d63-6e97a2edfaac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:28.338261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3d15d-44cc-4832-8d63-6e97a2edfaac None admin] RBAC: Adding query filter params (role.id=1249af5cf5054410a11b2d88c03fcd52, scope.project.id=836a895c1fd443928a00b4d27d10a8f9, user.id=8d32be869c6c48259625a2926009e24a) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:28.338603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3d15d-44cc-4832-8d63-6e97a2edfaac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:28.340052 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c3d15d-44cc-4832-8d63-6e97a2edfaac None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:28.342848 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819006d4-a1f4-4e56-8701-fa4a4684335e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:28.345927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 44/88] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:28 2022] GET /identity/v3/role_assignments?user.id=8d32be869c6c48259625a2926009e24a&scope.project.id=836a895c1fd443928a00b4d27d10a8f9&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:28.346743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3d15d-44cc-4832-8d63-6e97a2edfaac None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:28.383438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c8764-d250-4510-96d4-c06a1964eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:28.384393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c8764-d250-4510-96d4-c06a1964eaed None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:28.384675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c8764-d250-4510-96d4-c06a1964eaed None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:28.385020 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c8764-d250-4510-96d4-c06a1964eaed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:28.398498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:28.411824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:28.473467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c8764-d250-4510-96d4-c06a1964eaed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:28.474905 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c8764-d250-4510-96d4-c06a1964eaed None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:28.492457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c3d15d-44cc-4832-8d63-6e97a2edfaac None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:28.532155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dffa2-ce9d-4ba8-ba4c-fe50dc4c59b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:28.532977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dffa2-ce9d-4ba8-ba4c-fe50dc4c59b4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:28.533267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dffa2-ce9d-4ba8-ba4c-fe50dc4c59b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:28.533564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dffa2-ce9d-4ba8-ba4c-fe50dc4c59b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:28.596242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 45/89] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:28.597519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c8764-d250-4510-96d4-c06a1964eaed None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:28.627158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876dffa2-ce9d-4ba8-ba4c-fe50dc4c59b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:28.628970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876dffa2-ce9d-4ba8-ba4c-fe50dc4c59b4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:28.652455 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:28.674830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:28.721295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c8764-d250-4510-96d4-c06a1964eaed None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:28.743114 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 45/90] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:28.743869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dffa2-ce9d-4ba8-ba4c-fe50dc4c59b4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:28.762514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6158f60-3738-4f8d-af36-552bfa56f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:28.763519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6158f60-3738-4f8d-af36-552bfa56f9cb None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:28.763807 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6158f60-3738-4f8d-af36-552bfa56f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:28.764083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6158f60-3738-4f8d-af36-552bfa56f9cb None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:28.766168 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6158f60-3738-4f8d-af36-552bfa56f9cb None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:28.767649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6158f60-3738-4f8d-af36-552bfa56f9cb None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:28.769350 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 46/91] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:28 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:28.770111 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6158f60-3738-4f8d-af36-552bfa56f9cb None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:28.859060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dffa2-ce9d-4ba8-ba4c-fe50dc4c59b4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:28.883811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6158f60-3738-4f8d-af36-552bfa56f9cb None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:28.893813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:28.895146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183328f5-85a6-43b9-bda6-7eeec5a8a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:28.895803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183328f5-85a6-43b9-bda6-7eeec5a8a7fb None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:28.896101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183328f5-85a6-43b9-bda6-7eeec5a8a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:28.896428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183328f5-85a6-43b9-bda6-7eeec5a8a7fb None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:28.897170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183328f5-85a6-43b9-bda6-7eeec5a8a7fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:28.898664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183328f5-85a6-43b9-bda6-7eeec5a8a7fb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:28.907875 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:28.923263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 46/92] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:11:28 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:28.924286 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183328f5-85a6-43b9-bda6-7eeec5a8a7fb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:28.936193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bc362-e2f9-4cdf-9715-713167178b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:28.937255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bc362-e2f9-4cdf-9715-713167178b1d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:28.937648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bc362-e2f9-4cdf-9715-713167178b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:28.938030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bc362-e2f9-4cdf-9715-713167178b1d None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:28.939820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bc362-e2f9-4cdf-9715-713167178b1d None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:28.941394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bc362-e2f9-4cdf-9715-713167178b1d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:28.943721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 47/93] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:28 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:28.948926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bc362-e2f9-4cdf-9715-713167178b1d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:29.068913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bc362-e2f9-4cdf-9715-713167178b1d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:29.087209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183328f5-85a6-43b9-bda6-7eeec5a8a7fb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:29.126790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea52e6a-491a-44ee-b911-5f9d09d43102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:29.127339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff795e-31d1-4ac9-be4a-98efe44dcab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:29.128300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea52e6a-491a-44ee-b911-5f9d09d43102 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:29.128433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff795e-31d1-4ac9-be4a-98efe44dcab7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:29.128731 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff795e-31d1-4ac9-be4a-98efe44dcab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:29.129067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea52e6a-491a-44ee-b911-5f9d09d43102 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:29.129164 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff795e-31d1-4ac9-be4a-98efe44dcab7 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:29.129597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea52e6a-491a-44ee-b911-5f9d09d43102 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:29.130680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea52e6a-491a-44ee-b911-5f9d09d43102 None admin] RBAC: Adding query filter params (role.id=f4461b279e86422b83e29521e7ca29d0, scope.project.id=7139ad672c734cf799e4c39623384e9c, user.id=8d32be869c6c48259625a2926009e24a) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:29.130785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff795e-31d1-4ac9-be4a-98efe44dcab7 None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:29.131297 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea52e6a-491a-44ee-b911-5f9d09d43102 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:29.132475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff795e-31d1-4ac9-be4a-98efe44dcab7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:29.133174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea52e6a-491a-44ee-b911-5f9d09d43102 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:29.134492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 47/94] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:28 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:29.135372 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff795e-31d1-4ac9-be4a-98efe44dcab7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:29.144689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 48/95] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:28 2022] GET /identity/v3/role_assignments?user.id=8d32be869c6c48259625a2926009e24a&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=f4461b279e86422b83e29521e7ca29d0 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:29.145831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea52e6a-491a-44ee-b911-5f9d09d43102 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:29.203328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:29.234226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:29.288079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea52e6a-491a-44ee-b911-5f9d09d43102 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:29.290229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff795e-31d1-4ac9-be4a-98efe44dcab7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:29.337136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d00e1-f665-4f9f-8f0a-b757a77b252c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:29.337730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d00e1-f665-4f9f-8f0a-b757a77b252c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:29.338025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d00e1-f665-4f9f-8f0a-b757a77b252c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:29.338306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d00e1-f665-4f9f-8f0a-b757a77b252c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:29.339030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec2349-1470-4819-a33d-31b183f6dc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:29.339251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d00e1-f665-4f9f-8f0a-b757a77b252c None admin] RBAC: Adding query filter params (role.id=1249af5cf5054410a11b2d88c03fcd52, scope.project.id=7139ad672c734cf799e4c39623384e9c, user.id=8d32be869c6c48259625a2926009e24a) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:29.339573 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d00e1-f665-4f9f-8f0a-b757a77b252c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:29.340305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec2349-1470-4819-a33d-31b183f6dc45 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:29.340785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec2349-1470-4819-a33d-31b183f6dc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:29.341045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d00e1-f665-4f9f-8f0a-b757a77b252c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:29.341480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec2349-1470-4819-a33d-31b183f6dc45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:29.342735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec2349-1470-4819-a33d-31b183f6dc45 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=7139ad672c734cf799e4c39623384e9c, user.id=b868f066d8ff4579b72df10988c4ede1) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:29.343400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec2349-1470-4819-a33d-31b183f6dc45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:29.345108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec2349-1470-4819-a33d-31b183f6dc45 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:29.354657 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 48/96] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:29 2022] GET /identity/v3/role_assignments?user.id=8d32be869c6c48259625a2926009e24a&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:29.358383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 49/97] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:29 2022] GET /identity/v3/role_assignments?user.id=b868f066d8ff4579b72df10988c4ede1&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=d67707782358417ba9f25b2793ca2890 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:31.192275 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86014-fe00-4577-86d2-545eca264002 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:31.193316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86014-fe00-4577-86d2-545eca264002 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:31.193850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86014-fe00-4577-86d2-545eca264002 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:31.195512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 49/98] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:11:31.204567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e362-23cf-4efc-9c6e-b3113e105031 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:31.205472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e362-23cf-4efc-9c6e-b3113e105031 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:31.205954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e362-23cf-4efc-9c6e-b3113e105031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:31.225271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-4209e362-23cf-4efc-9c6e-b3113e105031 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:31.226962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-4209e362-23cf-4efc-9c6e-b3113e105031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:31.313620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 50/99] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:31.342093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b86014-fe00-4577-86d2-545eca264002 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:31.497264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b86014-fe00-4577-86d2-545eca264002 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:31.547080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:31.567758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adb6db-0a16-4f75-8bb1-d31a6002c93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:31.569151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adb6db-0a16-4f75-8bb1-d31a6002c93c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:31.569660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adb6db-0a16-4f75-8bb1-d31a6002c93c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:31.570146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adb6db-0a16-4f75-8bb1-d31a6002c93c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:31.596094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62adb6db-0a16-4f75-8bb1-d31a6002c93c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-62adb6db-0a16-4f75-8bb1-d31a6002c93c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:31.616467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:31.617470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:31.618283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:31.618895 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 50/100] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:11:31.625423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209e362-23cf-4efc-9c6e-b3113e105031 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:31.691082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:31.791197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209e362-23cf-4efc-9c6e-b3113e105031 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:31.839400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9713c5-8b92-4745-b3df-05c0e8ac91a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:31.840113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9713c5-8b92-4745-b3df-05c0e8ac91a0 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:31.840400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9713c5-8b92-4745-b3df-05c0e8ac91a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:31.840683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9713c5-8b92-4745-b3df-05c0e8ac91a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:31.841546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9713c5-8b92-4745-b3df-05c0e8ac91a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:31.841864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9713c5-8b92-4745-b3df-05c0e8ac91a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:31.843947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9713c5-8b92-4745-b3df-05c0e8ac91a0 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:31.855063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 51/101] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:31.867701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adb6db-0a16-4f75-8bb1-d31a6002c93c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:32.055934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adb6db-0a16-4f75-8bb1-d31a6002c93c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:32.098147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460527f-699b-4443-9eed-bb114af23c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:32.099081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460527f-699b-4443-9eed-bb114af23c9f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:32.099364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460527f-699b-4443-9eed-bb114af23c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:32.099644 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460527f-699b-4443-9eed-bb114af23c9f None admin] PATH_INFO: `/v3/users/8d32be869c6c48259625a2926009e24a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:32.100874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460527f-699b-4443-9eed-bb114af23c9f None admin] RBAC: Authorizing `identity:get_user(user_id=8d32be869c6c48259625a2926009e24a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:32.102346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460527f-699b-4443-9eed-bb114af23c9f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:32.104346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 51/102] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:31 2022] GET /identity/v3/users/8d32be869c6c48259625a2926009e24a => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:32.111652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9713c5-8b92-4745-b3df-05c0e8ac91a0 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:32.238946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:32.247256 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9713c5-8b92-4745-b3df-05c0e8ac91a0 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:32.295181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2157a7d5-e6a3-4343-a00a-1c6c8dabc253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:32.296075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157a7d5-e6a3-4343-a00a-1c6c8dabc253 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:32.296364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157a7d5-e6a3-4343-a00a-1c6c8dabc253 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:32.296641 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2157a7d5-e6a3-4343-a00a-1c6c8dabc253 None admin] PATH_INFO: `/v3/projects/836a895c1fd443928a00b4d27d10a8f9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:32.297859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157a7d5-e6a3-4343-a00a-1c6c8dabc253 None admin] RBAC: Authorizing `identity:get_project(project_id=836a895c1fd443928a00b4d27d10a8f9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:32.299354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2157a7d5-e6a3-4343-a00a-1c6c8dabc253 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:32.303215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 52/103] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:32 2022] GET /identity/v3/projects/836a895c1fd443928a00b4d27d10a8f9 => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:32.312756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460527f-699b-4443-9eed-bb114af23c9f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:32.439903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:32.508113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460527f-699b-4443-9eed-bb114af23c9f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:32.554145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ca52f-3019-4d3a-807a-8eeaaaf8e0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:32.555092 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ca52f-3019-4d3a-807a-8eeaaaf8e0d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:32.555381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ca52f-3019-4d3a-807a-8eeaaaf8e0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:32.555669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ca52f-3019-4d3a-807a-8eeaaaf8e0d1 None admin] PATH_INFO: `/v3/projects/836a895c1fd443928a00b4d27d10a8f9/users/8d32be869c6c48259625a2926009e24a/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:32.557872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ca52f-3019-4d3a-807a-8eeaaaf8e0d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=836a895c1fd443928a00b4d27d10a8f9, user_id=8d32be869c6c48259625a2926009e24a, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:32.559392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3ca52f-3019-4d3a-807a-8eeaaaf8e0d1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:32.570872 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81269-2332-4ead-a9ef-bea1c0e3e8b4 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:32.571152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81269-2332-4ead-a9ef-bea1c0e3e8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:32.571357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e81269-2332-4ead-a9ef-bea1c0e3e8b4 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:32.574056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 53/104] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:32.581852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59675b-c024-4e74-b8c7-05d880018991 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:32.582141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59675b-c024-4e74-b8c7-05d880018991 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:32.582644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59675b-c024-4e74-b8c7-05d880018991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:32.592241 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 52/105] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:11:32 2022] PUT /identity/v3/projects/836a895c1fd443928a00b4d27d10a8f9/users/8d32be869c6c48259625a2926009e24a/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:11:32.620448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-ef59675b-c024-4e74-b8c7-05d880018991 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:32.621657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ef59675b-c024-4e74-b8c7-05d880018991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:32.732339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 54/106] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:32.767568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ca52f-3019-4d3a-807a-8eeaaaf8e0d1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:32.859768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5e6c-c3d0-427a-a94f-f5186172fbe7 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:32.860772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5e6c-c3d0-427a-a94f-f5186172fbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:32.861575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5e6c-c3d0-427a-a94f-f5186172fbe7 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:32.864475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 55/107] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:32.873780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4fa94-465d-4ef5-addf-eb4b6d92ea3b None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:32.875380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4fa94-465d-4ef5-addf-eb4b6d92ea3b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:32.876137 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4fa94-465d-4ef5-addf-eb4b6d92ea3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:32.935676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-f6f4fa94-465d-4ef5-addf-eb4b6d92ea3b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:32.937835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f6f4fa94-465d-4ef5-addf-eb4b6d92ea3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:33.007392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ca52f-3019-4d3a-807a-8eeaaaf8e0d1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:33.039948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:33.080922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 56/108] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:33.086763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d6b60-55cb-45c9-b98e-0cfbf3c4e715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:33.087592 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d6b60-55cb-45c9-b98e-0cfbf3c4e715 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:33.087879 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d6b60-55cb-45c9-b98e-0cfbf3c4e715 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:33.088165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d6b60-55cb-45c9-b98e-0cfbf3c4e715 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:33.100213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d6b60-55cb-45c9-b98e-0cfbf3c4e715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-247d6b60-55cb-45c9-b98e-0cfbf3c4e715 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:33.109735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:33.110718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:33.111511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:33.112096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 53/109] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:11:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:33.118653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4fa94-465d-4ef5-addf-eb4b6d92ea3b None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:33.133525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d6b60-55cb-45c9-b98e-0cfbf3c4e715 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:33.166308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:33.266897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4fa94-465d-4ef5-addf-eb4b6d92ea3b None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:33.312454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a149e-8d08-415d-81b1-3eae8a234e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:33.313150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a149e-8d08-415d-81b1-3eae8a234e66 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:33.313432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a149e-8d08-415d-81b1-3eae8a234e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:33.313708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a149e-8d08-415d-81b1-3eae8a234e66 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:33.314480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a149e-8d08-415d-81b1-3eae8a234e66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:33.314817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a149e-8d08-415d-81b1-3eae8a234e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:33.316431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a149e-8d08-415d-81b1-3eae8a234e66 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:33.325829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 57/110] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:11:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:33.331942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a149e-8d08-415d-81b1-3eae8a234e66 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:33.424859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d6b60-55cb-45c9-b98e-0cfbf3c4e715 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:33.453852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:33.473821 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a149e-8d08-415d-81b1-3eae8a234e66 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:33.535096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea858d-7bcc-445f-afd1-d33e68b43e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:33.536207 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea858d-7bcc-445f-afd1-d33e68b43e74 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:33.536733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea858d-7bcc-445f-afd1-d33e68b43e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:33.537294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea858d-7bcc-445f-afd1-d33e68b43e74 None admin] PATH_INFO: `/v3/users/8d32be869c6c48259625a2926009e24a` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:33.539854 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea858d-7bcc-445f-afd1-d33e68b43e74 None admin] RBAC: Authorizing `identity:get_user(user_id=8d32be869c6c48259625a2926009e24a)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:33.541690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea858d-7bcc-445f-afd1-d33e68b43e74 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:33.543818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 58/111] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:33 2022] GET /identity/v3/users/8d32be869c6c48259625a2926009e24a => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:33.549230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f61211-3ddb-4181-8692-566ff675e7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:33.549732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea858d-7bcc-445f-afd1-d33e68b43e74 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:33.550580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f61211-3ddb-4181-8692-566ff675e7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:33.551051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f61211-3ddb-4181-8692-566ff675e7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:33.551511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f61211-3ddb-4181-8692-566ff675e7c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:33.587619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f61211-3ddb-4181-8692-566ff675e7c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-43f61211-3ddb-4181-8692-566ff675e7c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:33.601015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:33.601911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:33.602709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 54/112] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:11:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 469 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:11:33.609460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f61211-3ddb-4181-8692-566ff675e7c7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:33.690003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:33.740778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea858d-7bcc-445f-afd1-d33e68b43e74 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:33.787500 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040030e4-ee68-466e-acb1-2191bceb14dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:33.788629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040030e4-ee68-466e-acb1-2191bceb14dc None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:33.789082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040030e4-ee68-466e-acb1-2191bceb14dc None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:33.789500 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040030e4-ee68-466e-acb1-2191bceb14dc None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:33.791633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040030e4-ee68-466e-acb1-2191bceb14dc None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:33.794591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040030e4-ee68-466e-acb1-2191bceb14dc None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:33.797362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 59/113] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:33 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:33.805954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f500e-06b6-454c-ac7f-e45340942792 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:33.806428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f500e-06b6-454c-ac7f-e45340942792 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:33.806897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f500e-06b6-454c-ac7f-e45340942792 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:33.809188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 60/114] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:33.811304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835e87-9060-4689-a2a8-e2b7a22ec27a None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:33.812264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835e87-9060-4689-a2a8-e2b7a22ec27a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:33.812692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70835e87-9060-4689-a2a8-e2b7a22ec27a None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:33.815573 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 61/115] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:33.817097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70835e87-9060-4689-a2a8-e2b7a22ec27a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:33.849241 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f61211-3ddb-4181-8692-566ff675e7c7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:33.857897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:33.910679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968e806-ffef-47b2-b959-e070fb3def34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:33.912015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968e806-ffef-47b2-b959-e070fb3def34 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:33.912637 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968e806-ffef-47b2-b959-e070fb3def34 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:33.913389 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968e806-ffef-47b2-b959-e070fb3def34 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:33.914449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968e806-ffef-47b2-b959-e070fb3def34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:33.914927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968e806-ffef-47b2-b959-e070fb3def34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:33.918376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968e806-ffef-47b2-b959-e070fb3def34 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:33.934040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 55/116] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:33.935323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a75e06-09d7-47ea-9937-34e7264ac358 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:33.935624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a75e06-09d7-47ea-9937-34e7264ac358 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:33.935910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a75e06-09d7-47ea-9937-34e7264ac358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:33.966199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-91a75e06-09d7-47ea-9937-34e7264ac358 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:33.967752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-91a75e06-09d7-47ea-9937-34e7264ac358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:33.967962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70835e87-9060-4689-a2a8-e2b7a22ec27a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:34.006677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecf938-498c-4b3e-a836-63af68e8f2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:34.007634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf938-498c-4b3e-a836-63af68e8f2a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:34.007977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf938-498c-4b3e-a836-63af68e8f2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:34.008263 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecf938-498c-4b3e-a836-63af68e8f2a9 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c/users/8d32be869c6c48259625a2926009e24a/roles/f4461b279e86422b83e29521e7ca29d0` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:34.010641 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecf938-498c-4b3e-a836-63af68e8f2a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7139ad672c734cf799e4c39623384e9c, user_id=8d32be869c6c48259625a2926009e24a, role_id=f4461b279e86422b83e29521e7ca29d0)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:34.012274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecf938-498c-4b3e-a836-63af68e8f2a9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:34.029969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 62/117] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:11:33 2022] PUT /identity/v3/projects/7139ad672c734cf799e4c39623384e9c/users/8d32be869c6c48259625a2926009e24a/roles/f4461b279e86422b83e29521e7ca29d0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:11:34.031827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5838a6-cb91-45ac-93cd-291a4513b8c8 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:34.032193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5838a6-cb91-45ac-93cd-291a4513b8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:34.032586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5838a6-cb91-45ac-93cd-291a4513b8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:34.051996 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-cb5838a6-cb91-45ac-93cd-291a4513b8c8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:34.053506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-cb5838a6-cb91-45ac-93cd-291a4513b8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:34.053703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 56/118] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:34.055071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a75e06-09d7-47ea-9937-34e7264ac358 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:34.102991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:34.134868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 63/119] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:34.136688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32c469-1d36-4e45-ba11-27498c3621d5 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:34.137360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32c469-1d36-4e45-ba11-27498c3621d5 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:34.138062 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32c469-1d36-4e45-ba11-27498c3621d5 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:34.139732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 64/120] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:34.145145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32c469-1d36-4e45-ba11-27498c3621d5 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:34.169470 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:34.205232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a75e06-09d7-47ea-9937-34e7264ac358 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:34.246792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df915233-0feb-402e-af87-00eb80cd5b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:34.248373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df915233-0feb-402e-af87-00eb80cd5b0e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:34.250270 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df915233-0feb-402e-af87-00eb80cd5b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:34.250270 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df915233-0feb-402e-af87-00eb80cd5b0e None admin] PATH_INFO: `/v3/users/b868f066d8ff4579b72df10988c4ede1` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:34.252137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df915233-0feb-402e-af87-00eb80cd5b0e None admin] RBAC: Authorizing `identity:get_user(user_id=b868f066d8ff4579b72df10988c4ede1)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:34.255103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df915233-0feb-402e-af87-00eb80cd5b0e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:34.262577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 57/121] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:34 2022] GET /identity/v3/users/b868f066d8ff4579b72df10988c4ede1 => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:34.262577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4a2b6-9d2e-48bc-a904-6bc16da3f087 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:34.262577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4a2b6-9d2e-48bc-a904-6bc16da3f087 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:34.262577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4a2b6-9d2e-48bc-a904-6bc16da3f087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:34.289235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-75a4a2b6-9d2e-48bc-a904-6bc16da3f087 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:34.291385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-75a4a2b6-9d2e-48bc-a904-6bc16da3f087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:34.292710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32c469-1d36-4e45-ba11-27498c3621d5 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:34.340065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd69d9-cece-4628-be41-389718360700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:34.341313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd69d9-cece-4628-be41-389718360700 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:34.341947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd69d9-cece-4628-be41-389718360700 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:34.342422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dd69d9-cece-4628-be41-389718360700 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:34.355778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dd69d9-cece-4628-be41-389718360700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-b4dd69d9-cece-4628-be41-389718360700 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:34.365081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:34.366014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:34.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:34.368819 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 65/122] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:34.369859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd69d9-cece-4628-be41-389718360700 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:34.432982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 58/123] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:34.439433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4a2b6-9d2e-48bc-a904-6bc16da3f087 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:34.473740 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:34.491801 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:34.540452 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dd69d9-cece-4628-be41-389718360700 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:34.586861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4a2b6-9d2e-48bc-a904-6bc16da3f087 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:34.601309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c578a9e-f372-421a-b911-fdb0dec00694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:34.603080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c578a9e-f372-421a-b911-fdb0dec00694 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:34.603731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c578a9e-f372-421a-b911-fdb0dec00694 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:34.604100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c578a9e-f372-421a-b911-fdb0dec00694 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:34.605983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c578a9e-f372-421a-b911-fdb0dec00694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:34.608078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c578a9e-f372-421a-b911-fdb0dec00694 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:34.612030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 66/124] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:34.613339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c578a9e-f372-421a-b911-fdb0dec00694 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:34.641975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7891ae9-05f2-4a48-9d4f-05e8b0f7a9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:34.643637 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7891ae9-05f2-4a48-9d4f-05e8b0f7a9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:34.644139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7891ae9-05f2-4a48-9d4f-05e8b0f7a9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:34.644880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7891ae9-05f2-4a48-9d4f-05e8b0f7a9a1 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:34.647425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7891ae9-05f2-4a48-9d4f-05e8b0f7a9a1 None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:34.650398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7891ae9-05f2-4a48-9d4f-05e8b0f7a9a1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:34.654123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 59/125] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:34 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:34.663956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7891ae9-05f2-4a48-9d4f-05e8b0f7a9a1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:34.748595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c578a9e-f372-421a-b911-fdb0dec00694 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:34.798452 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0491bb0-53f1-405f-b61b-201cc6461970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:34.799165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0491bb0-53f1-405f-b61b-201cc6461970 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:34.799453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0491bb0-53f1-405f-b61b-201cc6461970 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:34.799734 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0491bb0-53f1-405f-b61b-201cc6461970 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:34.800493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0491bb0-53f1-405f-b61b-201cc6461970 None admin] RBAC: Adding query filter params (name=member) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:34.800805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0491bb0-53f1-405f-b61b-201cc6461970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:34.802210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0491bb0-53f1-405f-b61b-201cc6461970 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:34.816747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 67/126] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:34.817964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0491bb0-53f1-405f-b61b-201cc6461970 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:34.825447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:34.834871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:34.848383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7891ae9-05f2-4a48-9d4f-05e8b0f7a9a1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:34.892024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c017e6-106d-4d95-bc0d-d333993af6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:34.893067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c017e6-106d-4d95-bc0d-d333993af6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:34.893552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c017e6-106d-4d95-bc0d-d333993af6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:34.894003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c017e6-106d-4d95-bc0d-d333993af6a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:34.910348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c017e6-106d-4d95-bc0d-d333993af6a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-12c017e6-106d-4d95-bc0d-d333993af6a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:34.924707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:34.925834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:11:34.926796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:34.929615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 60/127] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:34.931112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c017e6-106d-4d95-bc0d-d333993af6a0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:35.003318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0491bb0-53f1-405f-b61b-201cc6461970 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:35.046775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d636b-83c1-4566-be43-0c6e43f3bf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:35.047448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d636b-83c1-4566-be43-0c6e43f3bf21 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:35.047729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d636b-83c1-4566-be43-0c6e43f3bf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:35.048001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d636b-83c1-4566-be43-0c6e43f3bf21 None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:35.048716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d636b-83c1-4566-be43-0c6e43f3bf21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:35.050173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d636b-83c1-4566-be43-0c6e43f3bf21 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:35.083707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 68/128] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:11:34 2022] POST /identity/v3/projects => generated 302 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:35.084513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d636b-83c1-4566-be43-0c6e43f3bf21 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:35.115284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c017e6-106d-4d95-bc0d-d333993af6a0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:35.152451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:35.207770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d636b-83c1-4566-be43-0c6e43f3bf21 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:35.242508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d19c57-941d-4e02-9295-414d773dd67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:35.244295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19c57-941d-4e02-9295-414d773dd67e None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:35.244802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19c57-941d-4e02-9295-414d773dd67e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:35.245234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d19c57-941d-4e02-9295-414d773dd67e None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c/users/b868f066d8ff4579b72df10988c4ede1/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:35.246401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f748de-f7c1-4e9b-a651-73995ded065d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:35.247384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f748de-f7c1-4e9b-a651-73995ded065d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:35.247739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f748de-f7c1-4e9b-a651-73995ded065d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:35.248083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f748de-f7c1-4e9b-a651-73995ded065d None admin] PATH_INFO: `/v3/users/8d32be869c6c48259625a2926009e24a` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:35.250036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f748de-f7c1-4e9b-a651-73995ded065d None admin] RBAC: Authorizing `identity:get_user(user_id=8d32be869c6c48259625a2926009e24a)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:35.251653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f748de-f7c1-4e9b-a651-73995ded065d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:35.253477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 69/129] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:35 2022] GET /identity/v3/users/8d32be869c6c48259625a2926009e24a => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:35.254339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f748de-f7c1-4e9b-a651-73995ded065d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:35.255146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d19c57-941d-4e02-9295-414d773dd67e None admin] RBAC: Authorizing `identity:create_grant(project_id=7139ad672c734cf799e4c39623384e9c, user_id=b868f066d8ff4579b72df10988c4ede1, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:35.258202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d19c57-941d-4e02-9295-414d773dd67e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:35.279856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 61/130] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:11:34 2022] PUT /identity/v3/projects/7139ad672c734cf799e4c39623384e9c/users/b868f066d8ff4579b72df10988c4ede1/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:11:35.291018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d19c57-941d-4e02-9295-414d773dd67e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:35.333058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:35.376683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f748de-f7c1-4e9b-a651-73995ded065d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:35.414940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:35.424892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdea6e-2f89-481c-9705-d3f42481bbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:35.425588 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdea6e-2f89-481c-9705-d3f42481bbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:35.425967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdea6e-2f89-481c-9705-d3f42481bbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:35.426254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdea6e-2f89-481c-9705-d3f42481bbe5 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:35.427169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdea6e-2f89-481c-9705-d3f42481bbe5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:35.427478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdea6e-2f89-481c-9705-d3f42481bbe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:35.428973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fdea6e-2f89-481c-9705-d3f42481bbe5 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:35.447562 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 70/131] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:35.454355 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdea6e-2f89-481c-9705-d3f42481bbe5 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:35.522789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d19c57-941d-4e02-9295-414d773dd67e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:35.583988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdea6e-2f89-481c-9705-d3f42481bbe5 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:35.623571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e24060-c5cf-4dd1-b947-e3ef10afe29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:35.624509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e24060-c5cf-4dd1-b947-e3ef10afe29b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:35.624797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e24060-c5cf-4dd1-b947-e3ef10afe29b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:35.625078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e24060-c5cf-4dd1-b947-e3ef10afe29b None admin] PATH_INFO: `/v3/users/e016498b552f4919a357ac5ec6defa37` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:35.626704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179ab10-27b8-457c-a962-5cde2f6eedf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:35.627612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179ab10-27b8-457c-a962-5cde2f6eedf1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:35.627891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179ab10-27b8-457c-a962-5cde2f6eedf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:35.628166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179ab10-27b8-457c-a962-5cde2f6eedf1 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:35.633489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179ab10-27b8-457c-a962-5cde2f6eedf1 None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:35.635028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179ab10-27b8-457c-a962-5cde2f6eedf1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:35.636851 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 62/132] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:35 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:35.645881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179ab10-27b8-457c-a962-5cde2f6eedf1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:35.665164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e24060-c5cf-4dd1-b947-e3ef10afe29b None admin] RBAC: Authorizing `identity:get_user(user_id=e016498b552f4919a357ac5ec6defa37)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:35.665164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e24060-c5cf-4dd1-b947-e3ef10afe29b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:35.665164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 71/133] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:35 2022] GET /identity/v3/users/e016498b552f4919a357ac5ec6defa37 => generated 289 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:35.677240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e24060-c5cf-4dd1-b947-e3ef10afe29b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:35.726281 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:35.806628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e24060-c5cf-4dd1-b947-e3ef10afe29b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:35.818311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:35.849011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349b2a6-c5a0-4935-be2e-4f4ceeb9d528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:35.849879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349b2a6-c5a0-4935-be2e-4f4ceeb9d528 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:35.850221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349b2a6-c5a0-4935-be2e-4f4ceeb9d528 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:35.850619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349b2a6-c5a0-4935-be2e-4f4ceeb9d528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:35.851781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179ab10-27b8-457c-a962-5cde2f6eedf1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:35.928316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349b2a6-c5a0-4935-be2e-4f4ceeb9d528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:35.929905 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349b2a6-c5a0-4935-be2e-4f4ceeb9d528 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:35.933472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01513d34-ddf5-4ada-8a68-54843aa010bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:35.934420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01513d34-ddf5-4ada-8a68-54843aa010bc None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:35.934724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01513d34-ddf5-4ada-8a68-54843aa010bc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:35.935013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01513d34-ddf5-4ada-8a68-54843aa010bc None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c/users/8d32be869c6c48259625a2926009e24a/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:35.937048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01513d34-ddf5-4ada-8a68-54843aa010bc None admin] RBAC: Authorizing `identity:create_grant(project_id=7139ad672c734cf799e4c39623384e9c, user_id=8d32be869c6c48259625a2926009e24a, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:35.938519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01513d34-ddf5-4ada-8a68-54843aa010bc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:35.961710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 63/134] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:11:35 2022] PUT /identity/v3/projects/7139ad672c734cf799e4c39623384e9c/users/8d32be869c6c48259625a2926009e24a/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:11:35.979282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:36.046341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 72/135] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:36.051824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01513d34-ddf5-4ada-8a68-54843aa010bc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:36.129984 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463884d-fc3e-4d02-ae82-116d15964e02 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:36.130387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463884d-fc3e-4d02-ae82-116d15964e02 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:36.130883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463884d-fc3e-4d02-ae82-116d15964e02 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:36.133482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 73/136] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:36.144322 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38c837-c924-43bb-8c2d-9da379a7c864 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:36.144862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38c837-c924-43bb-8c2d-9da379a7c864 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:36.145384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38c837-c924-43bb-8c2d-9da379a7c864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:36.188754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0f38c837-c924-43bb-8c2d-9da379a7c864 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:36.190646 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0f38c837-c924-43bb-8c2d-9da379a7c864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:36.301932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01513d34-ddf5-4ada-8a68-54843aa010bc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:36.318566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 74/137] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:36.396053 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2092894-b4a9-4f89-919b-57f3015dca2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:36.397866 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2092894-b4a9-4f89-919b-57f3015dca2a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:36.398646 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2092894-b4a9-4f89-919b-57f3015dca2a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:36.399371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2092894-b4a9-4f89-919b-57f3015dca2a None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:36.407905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2092894-b4a9-4f89-919b-57f3015dca2a None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:36.410952 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2092894-b4a9-4f89-919b-57f3015dca2a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:36.414522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 64/138] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:36 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:36.416752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38c837-c924-43bb-8c2d-9da379a7c864 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:36.421091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2092894-b4a9-4f89-919b-57f3015dca2a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:36.437656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:36.570819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2092894-b4a9-4f89-919b-57f3015dca2a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:36.581496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:36.615955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9a889-2e4e-46fa-afbf-556e73b73ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:36.616975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9a889-2e4e-46fa-afbf-556e73b73ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:36.617267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9a889-2e4e-46fa-afbf-556e73b73ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:36.617545 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9a889-2e4e-46fa-afbf-556e73b73ee2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:36.618398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9a889-2e4e-46fa-afbf-556e73b73ee2 None admin] RBAC: Adding query filter params (role.id=989320df8c4149cf918861e3ee5954d2, scope.project.id=7139ad672c734cf799e4c39623384e9c, user.id=e016498b552f4919a357ac5ec6defa37) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:36.618741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9a889-2e4e-46fa-afbf-556e73b73ee2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:36.620226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f9a889-2e4e-46fa-afbf-556e73b73ee2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:36.631176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 65/139] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:36 2022] GET /identity/v3/role_assignments?user.id=e016498b552f4919a357ac5ec6defa37&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=989320df8c4149cf918861e3ee5954d2 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:36.647969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38c837-c924-43bb-8c2d-9da379a7c864 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:36.707434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b993e1d-82aa-4354-ad4c-721d039c4041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:36.708317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b993e1d-82aa-4354-ad4c-721d039c4041 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:36.708607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b993e1d-82aa-4354-ad4c-721d039c4041 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:36.708887 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b993e1d-82aa-4354-ad4c-721d039c4041 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:36.727889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b993e1d-82aa-4354-ad4c-721d039c4041 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-8b993e1d-82aa-4354-ad4c-721d039c4041 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:36.748660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:36.749655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:36.750888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 75/140] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:36.759770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9a889-2e4e-46fa-afbf-556e73b73ee2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:36.885526 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:36.925551 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9a889-2e4e-46fa-afbf-556e73b73ee2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:36.963446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cbee6-ea3c-4a51-b583-faad97fb1a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:36.964133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cbee6-ea3c-4a51-b583-faad97fb1a38 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:36.964421 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cbee6-ea3c-4a51-b583-faad97fb1a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:36.964700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982cbee6-ea3c-4a51-b583-faad97fb1a38 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:36.965464 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cbee6-ea3c-4a51-b583-faad97fb1a38 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:36.965958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cbee6-ea3c-4a51-b583-faad97fb1a38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:36.967491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982cbee6-ea3c-4a51-b583-faad97fb1a38 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:36.977280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 66/141] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:36.987869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b993e1d-82aa-4354-ad4c-721d039c4041 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:37.147788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b993e1d-82aa-4354-ad4c-721d039c4041 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:37.192609 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:37.201609 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255884-52f5-4c24-b396-7367f30baa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:37.204122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255884-52f5-4c24-b396-7367f30baa6a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:37.204650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255884-52f5-4c24-b396-7367f30baa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:37.205122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255884-52f5-4c24-b396-7367f30baa6a None admin] PATH_INFO: `/v3/users/8d32be869c6c48259625a2926009e24a` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:37.211525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e255884-52f5-4c24-b396-7367f30baa6a None admin] RBAC: Authorizing `identity:get_user(user_id=8d32be869c6c48259625a2926009e24a)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:37.213433 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e255884-52f5-4c24-b396-7367f30baa6a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:37.215490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 76/142] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:36 2022] GET /identity/v3/users/8d32be869c6c48259625a2926009e24a => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:37.225067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cbee6-ea3c-4a51-b583-faad97fb1a38 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:37.370783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982cbee6-ea3c-4a51-b583-faad97fb1a38 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:37.417062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f5226-1739-4d4c-9c46-1aa6bd2c2fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:37.417726 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f5226-1739-4d4c-9c46-1aa6bd2c2fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:37.418013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f5226-1739-4d4c-9c46-1aa6bd2c2fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:37.418338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f5226-1739-4d4c-9c46-1aa6bd2c2fe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:37.421237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:37.495465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f5226-1739-4d4c-9c46-1aa6bd2c2fe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:37.496871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f5226-1739-4d4c-9c46-1aa6bd2c2fe9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:37.621535 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 67/143] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:37.624121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255884-52f5-4c24-b396-7367f30baa6a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:37.828857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255884-52f5-4c24-b396-7367f30baa6a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:37.877927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b3fd5-45fc-4ae4-b972-833fb2f6e952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:37.878828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b3fd5-45fc-4ae4-b972-833fb2f6e952 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:37.879113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b3fd5-45fc-4ae4-b972-833fb2f6e952 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:37.879463 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b3fd5-45fc-4ae4-b972-833fb2f6e952 None admin] PATH_INFO: `/v3/projects/836a895c1fd443928a00b4d27d10a8f9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:37.881262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b3fd5-45fc-4ae4-b972-833fb2f6e952 None admin] RBAC: Authorizing `identity:get_project(project_id=836a895c1fd443928a00b4d27d10a8f9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:37.883594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b3fd5-45fc-4ae4-b972-833fb2f6e952 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:37.886387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 77/144] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:37 2022] GET /identity/v3/projects/836a895c1fd443928a00b4d27d10a8f9 => generated 312 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:37.893230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f5226-1739-4d4c-9c46-1aa6bd2c2fe9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:37.947472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:37.968623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a673384-ef33-4ff0-ad79-aa1cd9207c87 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:37.969231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a673384-ef33-4ff0-ad79-aa1cd9207c87 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:37.969708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a673384-ef33-4ff0-ad79-aa1cd9207c87 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:37.971519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 78/145] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:37.989906 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d4eaa-20c9-4eee-9ccc-0a55af8e7fbe None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:37.990750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d4eaa-20c9-4eee-9ccc-0a55af8e7fbe None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:37.991482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d4eaa-20c9-4eee-9ccc-0a55af8e7fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:38.031310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f5226-1739-4d4c-9c46-1aa6bd2c2fe9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:38.040751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d06d4eaa-20c9-4eee-9ccc-0a55af8e7fbe None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:38.043026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d06d4eaa-20c9-4eee-9ccc-0a55af8e7fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:38.081493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35273105-c554-4fcc-8953-96cf93a8ac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:38.082413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35273105-c554-4fcc-8953-96cf93a8ac57 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:38.082900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35273105-c554-4fcc-8953-96cf93a8ac57 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:38.083318 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35273105-c554-4fcc-8953-96cf93a8ac57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:38.084300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35273105-c554-4fcc-8953-96cf93a8ac57 None admin] RBAC: Adding query filter params (role.id=1249af5cf5054410a11b2d88c03fcd52, scope.project.id=836a895c1fd443928a00b4d27d10a8f9, user.id=8d32be869c6c48259625a2926009e24a) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:38.084751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35273105-c554-4fcc-8953-96cf93a8ac57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:38.086340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35273105-c554-4fcc-8953-96cf93a8ac57 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:38.092916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 68/146] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:37 2022] GET /identity/v3/role_assignments?user.id=8d32be869c6c48259625a2926009e24a&scope.project.id=836a895c1fd443928a00b4d27d10a8f9&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:38.171379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:38.177626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 79/147] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:38.216826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35273105-c554-4fcc-8953-96cf93a8ac57 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:38.341143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:38.350894 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35273105-c554-4fcc-8953-96cf93a8ac57 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:38.395545 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd44269-cbc4-4972-926e-e7b977b23487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:38.396399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd44269-cbc4-4972-926e-e7b977b23487 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:38.396683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd44269-cbc4-4972-926e-e7b977b23487 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:38.396956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd44269-cbc4-4972-926e-e7b977b23487 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:38.408859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd44269-cbc4-4972-926e-e7b977b23487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8cd44269-cbc4-4972-926e-e7b977b23487 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:38.423769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:38.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:38.425418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:38.426000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 69/148] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:11:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:38.431918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d4eaa-20c9-4eee-9ccc-0a55af8e7fbe None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:38.607632 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d4eaa-20c9-4eee-9ccc-0a55af8e7fbe None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:38.650166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cfc4d-7306-4909-8f0a-a6b5a014b8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:38.651639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cfc4d-7306-4909-8f0a-a6b5a014b8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:38.652096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cfc4d-7306-4909-8f0a-a6b5a014b8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:38.652594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cfc4d-7306-4909-8f0a-a6b5a014b8b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:38.654172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cfc4d-7306-4909-8f0a-a6b5a014b8b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:38.654740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cfc4d-7306-4909-8f0a-a6b5a014b8b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:38.656954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4cfc4d-7306-4909-8f0a-a6b5a014b8b8 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:38.666888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 80/149] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:11:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:38.673255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd44269-cbc4-4972-926e-e7b977b23487 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:38.803480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26453ffb-cc3d-4018-939e-8862d9662438 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:38.804077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26453ffb-cc3d-4018-939e-8862d9662438 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:38.804536 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26453ffb-cc3d-4018-939e-8862d9662438 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:38.806304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 81/150] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:38.813282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bccf63-9aff-4680-a8d2-324dc2d23f68 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:38.813818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bccf63-9aff-4680-a8d2-324dc2d23f68 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:38.814314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bccf63-9aff-4680-a8d2-324dc2d23f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:38.822981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd44269-cbc4-4972-926e-e7b977b23487 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:38.851582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d1bccf63-9aff-4680-a8d2-324dc2d23f68 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:38.853013 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d1bccf63-9aff-4680-a8d2-324dc2d23f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:38.881272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4f5e4-6f22-43c1-be8c-e92b39c3447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:38.882084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4f5e4-6f22-43c1-be8c-e92b39c3447f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:38.882548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4f5e4-6f22-43c1-be8c-e92b39c3447f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:38.882963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4f5e4-6f22-43c1-be8c-e92b39c3447f None admin] PATH_INFO: `/v3/users/8d32be869c6c48259625a2926009e24a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:38.891405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4f5e4-6f22-43c1-be8c-e92b39c3447f None admin] RBAC: Authorizing `identity:get_user(user_id=8d32be869c6c48259625a2926009e24a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:38.893111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4f5e4-6f22-43c1-be8c-e92b39c3447f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:38.894862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 70/151] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:38 2022] GET /identity/v3/users/8d32be869c6c48259625a2926009e24a => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:38.901231 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4f5e4-6f22-43c1-be8c-e92b39c3447f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:38.935017 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:38.945153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:38.956702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 82/152] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:38.984771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bccf63-9aff-4680-a8d2-324dc2d23f68 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:39.078736 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4f5e4-6f22-43c1-be8c-e92b39c3447f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:39.157530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdadc8fa-7bc4-4d0f-a726-b53335a41250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:39.158562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadc8fa-7bc4-4d0f-a726-b53335a41250 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:39.159122 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadc8fa-7bc4-4d0f-a726-b53335a41250 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:39.160325 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadc8fa-7bc4-4d0f-a726-b53335a41250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:39.193934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bccf63-9aff-4680-a8d2-324dc2d23f68 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:39.280788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dc491-322c-4f51-843b-308a82612c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:39.281607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dc491-322c-4f51-843b-308a82612c6a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:39.281885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dc491-322c-4f51-843b-308a82612c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:39.282154 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dc491-322c-4f51-843b-308a82612c6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:39.307000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865dc491-322c-4f51-843b-308a82612c6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:39.316094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadc8fa-7bc4-4d0f-a726-b53335a41250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-865dc491-322c-4f51-843b-308a82612c6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:39.317737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:39.318693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:39.320229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:39.321228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadc8fa-7bc4-4d0f-a726-b53335a41250 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:39.323378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 83/153] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:11:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:11:39.331052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dc491-322c-4f51-843b-308a82612c6a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:39.392361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:39.423955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:39.484211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dc491-322c-4f51-843b-308a82612c6a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:39.522584 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce4f4-1976-43e4-a6bc-057fb2153669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:39.523625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce4f4-1976-43e4-a6bc-057fb2153669 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:39.524109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce4f4-1976-43e4-a6bc-057fb2153669 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:39.524616 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ce4f4-1976-43e4-a6bc-057fb2153669 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:39.525571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ce4f4-1976-43e4-a6bc-057fb2153669 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:39.526076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ce4f4-1976-43e4-a6bc-057fb2153669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:39.527703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65ce4f4-1976-43e4-a6bc-057fb2153669 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:39.537597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 84/154] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:39.543733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce4f4-1976-43e4-a6bc-057fb2153669 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:39.574077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 71/155] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:39.580413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdadc8fa-7bc4-4d0f-a726-b53335a41250 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:39.658125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ce4f4-1976-43e4-a6bc-057fb2153669 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:39.682325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:39.716147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280538ff-0613-4b6c-972e-22487d1b8d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:39.717423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280538ff-0613-4b6c-972e-22487d1b8d62 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:39.718456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280538ff-0613-4b6c-972e-22487d1b8d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:39.718981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280538ff-0613-4b6c-972e-22487d1b8d62 None admin] PATH_INFO: `/v3/users/b868f066d8ff4579b72df10988c4ede1` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:39.720572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280538ff-0613-4b6c-972e-22487d1b8d62 None admin] RBAC: Authorizing `identity:get_user(user_id=b868f066d8ff4579b72df10988c4ede1)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:39.722355 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280538ff-0613-4b6c-972e-22487d1b8d62 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:39.724412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 85/156] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:39 2022] GET /identity/v3/users/b868f066d8ff4579b72df10988c4ede1 => generated 247 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:39.727015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed9ec-657c-4e0a-a3fa-7749d43beff4 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:39.727576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed9ec-657c-4e0a-a3fa-7749d43beff4 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:39.731313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aed9ec-657c-4e0a-a3fa-7749d43beff4 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:39.734569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 86/157] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:39.735244 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aed9ec-657c-4e0a-a3fa-7749d43beff4 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:39.758311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdadc8fa-7bc4-4d0f-a726-b53335a41250 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:39.798788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:39.808765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230c62f-36d7-45e8-b9a1-f4021081c0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:39.809658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c62f-36d7-45e8-b9a1-f4021081c0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:39.809944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c62f-36d7-45e8-b9a1-f4021081c0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:39.810252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230c62f-36d7-45e8-b9a1-f4021081c0d1 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:39.812013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230c62f-36d7-45e8-b9a1-f4021081c0d1 None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:39.813491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230c62f-36d7-45e8-b9a1-f4021081c0d1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:39.815244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 72/158] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:39 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:39.816582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e32723-315d-4dd3-89c6-c1c0c3e4f3ec None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:39.816871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e32723-315d-4dd3-89c6-c1c0c3e4f3ec None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:39.817154 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e32723-315d-4dd3-89c6-c1c0c3e4f3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:39.837509 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f9e32723-315d-4dd3-89c6-c1c0c3e4f3ec None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:39.838741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f9e32723-315d-4dd3-89c6-c1c0c3e4f3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:39.869277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aed9ec-657c-4e0a-a3fa-7749d43beff4 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:39.914046 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa018762-8146-4396-ac95-2b3f6e038ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:39.914725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018762-8146-4396-ac95-2b3f6e038ebf None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:39.915015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018762-8146-4396-ac95-2b3f6e038ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:39.915300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018762-8146-4396-ac95-2b3f6e038ebf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:39.951897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 73/159] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:39.953946 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b31ee-49fb-4b81-b084-984c3511f592 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:39.954236 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b31ee-49fb-4b81-b084-984c3511f592 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:39.954515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b31ee-49fb-4b81-b084-984c3511f592 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:39.955853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 74/160] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:39.956551 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b31ee-49fb-4b81-b084-984c3511f592 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:39.996060 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:40.024214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa018762-8146-4396-ac95-2b3f6e038ebf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:40.025731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa018762-8146-4396-ac95-2b3f6e038ebf None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:40.120137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b31ee-49fb-4b81-b084-984c3511f592 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:40.147242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:40.160873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21780f-e0a1-4dae-98ae-7d10744793b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:40.161703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21780f-e0a1-4dae-98ae-7d10744793b7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:40.162057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21780f-e0a1-4dae-98ae-7d10744793b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:40.162391 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21780f-e0a1-4dae-98ae-7d10744793b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:40.163446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21780f-e0a1-4dae-98ae-7d10744793b7 None admin] RBAC: Adding query filter params (role.id=f4461b279e86422b83e29521e7ca29d0, scope.project.id=7139ad672c734cf799e4c39623384e9c, user.id=8d32be869c6c48259625a2926009e24a) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:40.163966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21780f-e0a1-4dae-98ae-7d10744793b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:40.165959 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21780f-e0a1-4dae-98ae-7d10744793b7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:40.173577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 75/161] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:39 2022] GET /identity/v3/role_assignments?user.id=8d32be869c6c48259625a2926009e24a&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=f4461b279e86422b83e29521e7ca29d0 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:40.173774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 87/162] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:40.174919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa018762-8146-4396-ac95-2b3f6e038ebf None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:40.175299 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de3b22-e986-42e3-8186-e2a0d1c8bc2a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:40.175723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de3b22-e986-42e3-8186-e2a0d1c8bc2a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:40.176123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de3b22-e986-42e3-8186-e2a0d1c8bc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:40.201031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-56de3b22-e986-42e3-8186-e2a0d1c8bc2a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:40.203309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-56de3b22-e986-42e3-8186-e2a0d1c8bc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:40.290439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:40.303170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 76/163] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:40.305070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adb836-2eb5-4aa5-a500-c8076309381c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:40.305366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adb836-2eb5-4aa5-a500-c8076309381c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:40.305639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adb836-2eb5-4aa5-a500-c8076309381c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:40.307152 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 77/164] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:40.307776 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adb836-2eb5-4aa5-a500-c8076309381c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:40.325249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa018762-8146-4396-ac95-2b3f6e038ebf None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:40.374855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61786e0c-15bb-4ce9-8c6f-4ea8771f20fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:40.375668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61786e0c-15bb-4ce9-8c6f-4ea8771f20fb None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:40.375938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61786e0c-15bb-4ce9-8c6f-4ea8771f20fb None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:40.376210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61786e0c-15bb-4ce9-8c6f-4ea8771f20fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:40.377424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61786e0c-15bb-4ce9-8c6f-4ea8771f20fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:40.378903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61786e0c-15bb-4ce9-8c6f-4ea8771f20fb None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:40.380528 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 88/165] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:40.381681 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eafc3-09d6-47f0-9690-90540c659e0e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:40.381959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eafc3-09d6-47f0-9690-90540c659e0e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:40.382223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370eafc3-09d6-47f0-9690-90540c659e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:40.403572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-370eafc3-09d6-47f0-9690-90540c659e0e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:40.404717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-370eafc3-09d6-47f0-9690-90540c659e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:40.476361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adb836-2eb5-4aa5-a500-c8076309381c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:40.493562 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 89/166] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:40.495779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370eafc3-09d6-47f0-9690-90540c659e0e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:40.506747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:40.529530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009671d-21c8-4e82-8a04-6f17141a9e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:40.530949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009671d-21c8-4e82-8a04-6f17141a9e83 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:40.531232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009671d-21c8-4e82-8a04-6f17141a9e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:40.531513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009671d-21c8-4e82-8a04-6f17141a9e83 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:40.532717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009671d-21c8-4e82-8a04-6f17141a9e83 None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:40.534184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009671d-21c8-4e82-8a04-6f17141a9e83 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:40.535922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 78/167] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:40 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:40.540941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009671d-21c8-4e82-8a04-6f17141a9e83 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:40.610111 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:40.645102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370eafc3-09d6-47f0-9690-90540c659e0e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:40.681813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009671d-21c8-4e82-8a04-6f17141a9e83 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:40.697115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99352b10-1105-4b15-9392-5e902941b9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:40.698006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99352b10-1105-4b15-9392-5e902941b9fc None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:40.698290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99352b10-1105-4b15-9392-5e902941b9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:40.698589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99352b10-1105-4b15-9392-5e902941b9fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:40.713918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99352b10-1105-4b15-9392-5e902941b9fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-99352b10-1105-4b15-9392-5e902941b9fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:40.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:40.729337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:40.730131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 90/168] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:40.730878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99352b10-1105-4b15-9392-5e902941b9fc None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:40.731935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc1960-57e1-451b-9d2c-2aa2e5a7bc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:40.732685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc1960-57e1-451b-9d2c-2aa2e5a7bc9f None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:40.733023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc1960-57e1-451b-9d2c-2aa2e5a7bc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:40.733381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc1960-57e1-451b-9d2c-2aa2e5a7bc9f None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:40.734142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc1960-57e1-451b-9d2c-2aa2e5a7bc9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:40.735690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc1960-57e1-451b-9d2c-2aa2e5a7bc9f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:40.770702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 79/169] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:11:40 2022] POST /identity/v3/users => generated 283 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:40.770702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc1960-57e1-451b-9d2c-2aa2e5a7bc9f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:40.821528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:40.901718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99352b10-1105-4b15-9392-5e902941b9fc None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:40.941744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc1960-57e1-451b-9d2c-2aa2e5a7bc9f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:40.966112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce3387-62f2-4f19-93aa-693992546ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:40.967563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce3387-62f2-4f19-93aa-693992546ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:40.968065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce3387-62f2-4f19-93aa-693992546ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:40.968527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce3387-62f2-4f19-93aa-693992546ce0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:40.981247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:40.988856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce3387-62f2-4f19-93aa-693992546ce0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-35ce3387-62f2-4f19-93aa-693992546ce0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:40.996763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:40.998182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 91/170] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:41.000247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce3387-62f2-4f19-93aa-693992546ce0 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:41.009434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e1473-5d3c-48af-a08b-c1a452401d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:41.010288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e1473-5d3c-48af-a08b-c1a452401d18 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:41.010832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e1473-5d3c-48af-a08b-c1a452401d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:41.011274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e1473-5d3c-48af-a08b-c1a452401d18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:41.012389 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e1473-5d3c-48af-a08b-c1a452401d18 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=7139ad672c734cf799e4c39623384e9c, user.id=b868f066d8ff4579b72df10988c4ede1) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:41.013229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e1473-5d3c-48af-a08b-c1a452401d18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:41.015406 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e1473-5d3c-48af-a08b-c1a452401d18 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:41.026353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 80/171] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:40 2022] GET /identity/v3/role_assignments?user.id=b868f066d8ff4579b72df10988c4ede1&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=d67707782358417ba9f25b2793ca2890 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:41.028042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e1473-5d3c-48af-a08b-c1a452401d18 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:41.189800 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce3387-62f2-4f19-93aa-693992546ce0 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:41.196664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e1473-5d3c-48af-a08b-c1a452401d18 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:41.204341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:41.253496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57fa69-ea61-4c49-a3ba-d00e7fc9c9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:41.254687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57fa69-ea61-4c49-a3ba-d00e7fc9c9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:41.255095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57fa69-ea61-4c49-a3ba-d00e7fc9c9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:41.255394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c57fa69-ea61-4c49-a3ba-d00e7fc9c9d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:41.256344 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c57fa69-ea61-4c49-a3ba-d00e7fc9c9d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:41.256694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c57fa69-ea61-4c49-a3ba-d00e7fc9c9d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:41.258149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c57fa69-ea61-4c49-a3ba-d00e7fc9c9d7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:41.272006 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 81/172] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:41.280714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ed778-0e5f-4d0c-a874-ab176e0591c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:41.281624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ed778-0e5f-4d0c-a874-ab176e0591c9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:41.282169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ed778-0e5f-4d0c-a874-ab176e0591c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:41.282374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ed778-0e5f-4d0c-a874-ab176e0591c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:41.283190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ed778-0e5f-4d0c-a874-ab176e0591c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:41.283505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ed778-0e5f-4d0c-a874-ab176e0591c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:41.283905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57fa69-ea61-4c49-a3ba-d00e7fc9c9d7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:41.284940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ed778-0e5f-4d0c-a874-ab176e0591c9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:41.297607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 92/173] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:41.305093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ed778-0e5f-4d0c-a874-ab176e0591c9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:41.422811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:41.453054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c57fa69-ea61-4c49-a3ba-d00e7fc9c9d7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:41.472552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ed778-0e5f-4d0c-a874-ab176e0591c9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:41.515063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d669317-b732-48cd-848f-e8a71c587561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:41.516683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d669317-b732-48cd-848f-e8a71c587561 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:41.517163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d669317-b732-48cd-848f-e8a71c587561 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:41.517629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d669317-b732-48cd-848f-e8a71c587561 None admin] PATH_INFO: `/v3/users/e016498b552f4919a357ac5ec6defa37` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:41.518246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f27ad-848d-4b2c-bbf5-2a16ff5a98e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:41.519652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d669317-b732-48cd-848f-e8a71c587561 None admin] RBAC: Authorizing `identity:get_user(user_id=e016498b552f4919a357ac5ec6defa37)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:41.519834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f27ad-848d-4b2c-bbf5-2a16ff5a98e4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:41.520330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f27ad-848d-4b2c-bbf5-2a16ff5a98e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:41.520757 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f27ad-848d-4b2c-bbf5-2a16ff5a98e4 None admin] PATH_INFO: `/v3/users/8d32be869c6c48259625a2926009e24a` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:41.522258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d669317-b732-48cd-848f-e8a71c587561 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:41.522837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f27ad-848d-4b2c-bbf5-2a16ff5a98e4 None admin] RBAC: Authorizing `identity:get_user(user_id=8d32be869c6c48259625a2926009e24a)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:41.525378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 82/174] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:41 2022] GET /identity/v3/users/e016498b552f4919a357ac5ec6defa37 => generated 289 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:41.530811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f27ad-848d-4b2c-bbf5-2a16ff5a98e4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:41.534047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 93/175] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:41 2022] GET /identity/v3/users/8d32be869c6c48259625a2926009e24a => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:41.535802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d669317-b732-48cd-848f-e8a71c587561 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:41.541601 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f27ad-848d-4b2c-bbf5-2a16ff5a98e4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:41.602974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:41.732103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d669317-b732-48cd-848f-e8a71c587561 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:41.769366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f27ad-848d-4b2c-bbf5-2a16ff5a98e4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:41.820227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:41.824650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26addaa9-fa41-4f0b-85fb-aeb0b82b96b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:41.827528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26addaa9-fa41-4f0b-85fb-aeb0b82b96b3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:41.828510 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26addaa9-fa41-4f0b-85fb-aeb0b82b96b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:41.829321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26addaa9-fa41-4f0b-85fb-aeb0b82b96b3 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:41.831947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26addaa9-fa41-4f0b-85fb-aeb0b82b96b3 None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:41.835079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26addaa9-fa41-4f0b-85fb-aeb0b82b96b3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:41.839639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 83/176] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:41 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:41.850024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e497b9b-bafd-45ee-9afb-ce4900768c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:41.851095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e497b9b-bafd-45ee-9afb-ce4900768c5b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:41.851564 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e497b9b-bafd-45ee-9afb-ce4900768c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:41.852034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e497b9b-bafd-45ee-9afb-ce4900768c5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:41.852702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26addaa9-fa41-4f0b-85fb-aeb0b82b96b3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:42.042582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e497b9b-bafd-45ee-9afb-ce4900768c5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:42.044580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e497b9b-bafd-45ee-9afb-ce4900768c5b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:42.102366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26addaa9-fa41-4f0b-85fb-aeb0b82b96b3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:42.104760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:42.149259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f702f5-23f1-47a3-bded-244b826976b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:42.150379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f702f5-23f1-47a3-bded-244b826976b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:42.150692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f702f5-23f1-47a3-bded-244b826976b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:42.152085 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f702f5-23f1-47a3-bded-244b826976b9 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c/users/e016498b552f4919a357ac5ec6defa37/roles/989320df8c4149cf918861e3ee5954d2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:42.156854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f702f5-23f1-47a3-bded-244b826976b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7139ad672c734cf799e4c39623384e9c, user_id=e016498b552f4919a357ac5ec6defa37, role_id=989320df8c4149cf918861e3ee5954d2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:42.158631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f702f5-23f1-47a3-bded-244b826976b9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:42.180437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 84/177] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:11:41 2022] PUT /identity/v3/projects/7139ad672c734cf799e4c39623384e9c/users/e016498b552f4919a357ac5ec6defa37/roles/989320df8c4149cf918861e3ee5954d2 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:11:42.249205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 94/178] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 708 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:42.254767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f702f5-23f1-47a3-bded-244b826976b9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:42.400991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f702f5-23f1-47a3-bded-244b826976b9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:42.456439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b13dcc-f199-4f53-8202-c34ed6e2ce40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:42.457335 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b13dcc-f199-4f53-8202-c34ed6e2ce40 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:42.457612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b13dcc-f199-4f53-8202-c34ed6e2ce40 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:42.457885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b13dcc-f199-4f53-8202-c34ed6e2ce40 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:42.463630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b13dcc-f199-4f53-8202-c34ed6e2ce40 None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:42.465141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b13dcc-f199-4f53-8202-c34ed6e2ce40 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:42.467013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 85/179] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:42 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:42.477113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e497b9b-bafd-45ee-9afb-ce4900768c5b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:42.528159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:42.682576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e497b9b-bafd-45ee-9afb-ce4900768c5b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:42.763933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651ea2f-4edb-48eb-bb32-dbdf3c738465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:42.765662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651ea2f-4edb-48eb-bb32-dbdf3c738465 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:42.766166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651ea2f-4edb-48eb-bb32-dbdf3c738465 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:42.766697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651ea2f-4edb-48eb-bb32-dbdf3c738465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:42.768456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651ea2f-4edb-48eb-bb32-dbdf3c738465 None admin] RBAC: Adding query filter params (role.id=1249af5cf5054410a11b2d88c03fcd52, scope.project.id=7139ad672c734cf799e4c39623384e9c, user.id=8d32be869c6c48259625a2926009e24a) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:42.768996 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651ea2f-4edb-48eb-bb32-dbdf3c738465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:42.771983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651ea2f-4edb-48eb-bb32-dbdf3c738465 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:42.787420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 95/180] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:42 2022] GET /identity/v3/role_assignments?user.id=8d32be869c6c48259625a2926009e24a&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:44.266442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2d7cb-1b4b-4dde-82a9-f53c4ccf801c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:44.267316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2d7cb-1b4b-4dde-82a9-f53c4ccf801c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:44.267316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2d7cb-1b4b-4dde-82a9-f53c4ccf801c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:44.269080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 86/181] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:44.276476 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3bf51-6899-4c5a-9aab-b2ec642ae128 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:44.277147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3bf51-6899-4c5a-9aab-b2ec642ae128 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:44.277273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e3bf51-6899-4c5a-9aab-b2ec642ae128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:44.303931 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-41e3bf51-6899-4c5a-9aab-b2ec642ae128 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:44.305680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-41e3bf51-6899-4c5a-9aab-b2ec642ae128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:44.409769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 96/182] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:44.442763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2d7cb-1b4b-4dde-82a9-f53c4ccf801c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:44.500532 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:44.617492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2d7cb-1b4b-4dde-82a9-f53c4ccf801c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:44.674560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be054e-814b-45be-ac15-2b502495a84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:44.674560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be054e-814b-45be-ac15-2b502495a84d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:44.674560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be054e-814b-45be-ac15-2b502495a84d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:44.674560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be054e-814b-45be-ac15-2b502495a84d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:44.693732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be054e-814b-45be-ac15-2b502495a84d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-60be054e-814b-45be-ac15-2b502495a84d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:44.711713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:44.713511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:44.715157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 87/183] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:11:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:11:44.720795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3bf51-6899-4c5a-9aab-b2ec642ae128 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:44.746938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:44.852052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e3bf51-6899-4c5a-9aab-b2ec642ae128 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:44.900052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54050a42-1174-44a6-a28c-ea585c89ceb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:44.901064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050a42-1174-44a6-a28c-ea585c89ceb5 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:44.901557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050a42-1174-44a6-a28c-ea585c89ceb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:44.902026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54050a42-1174-44a6-a28c-ea585c89ceb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:44.903280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54050a42-1174-44a6-a28c-ea585c89ceb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:44.903794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54050a42-1174-44a6-a28c-ea585c89ceb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:44.906006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54050a42-1174-44a6-a28c-ea585c89ceb5 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:44.920167 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 97/184] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:44.931581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be054e-814b-45be-ac15-2b502495a84d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:44.940126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1deea8-dd1f-4a64-aa66-002c86e47c6c None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:44.940413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1deea8-dd1f-4a64-aa66-002c86e47c6c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:44.940859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1deea8-dd1f-4a64-aa66-002c86e47c6c None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:44.942386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 98/185] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:11:44.947678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f536236-b14d-4759-aac7-3ae64913222c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:44.948173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f536236-b14d-4759-aac7-3ae64913222c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:44.948627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f536236-b14d-4759-aac7-3ae64913222c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:44.966276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-1f536236-b14d-4759-aac7-3ae64913222c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:44.968305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1f536236-b14d-4759-aac7-3ae64913222c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:45.040640 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:45.052655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be054e-814b-45be-ac15-2b502495a84d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:45.055201 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 99/186] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:45.056569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6965fca-c330-452f-a04e-b63e3c802acc None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:45.056900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6965fca-c330-452f-a04e-b63e3c802acc None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:45.057235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6965fca-c330-452f-a04e-b63e3c802acc None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:45.058551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 100/187] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:45.059814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81f18d-0864-4a4a-baf6-e498acd43cc6 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:45.060242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81f18d-0864-4a4a-baf6-e498acd43cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:45.060660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81f18d-0864-4a4a-baf6-e498acd43cc6 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:45.062674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 101/188] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:45.066849 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee84ced-3e79-45c8-8ab1-11562a5489d1 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:45.067346 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee84ced-3e79-45c8-8ab1-11562a5489d1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:45.067758 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee84ced-3e79-45c8-8ab1-11562a5489d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:45.094521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-bee84ced-3e79-45c8-8ab1-11562a5489d1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:45.095756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-bee84ced-3e79-45c8-8ab1-11562a5489d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:45.106784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:45.113095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95076d4-f2a9-4601-9450-7a0b2a46e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:45.114599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95076d4-f2a9-4601-9450-7a0b2a46e975 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:45.115088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95076d4-f2a9-4601-9450-7a0b2a46e975 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:45.115533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95076d4-f2a9-4601-9450-7a0b2a46e975 None admin] PATH_INFO: `/v3/users/83b8dc3f532542ddbe8018d329327c24` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:45.134174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95076d4-f2a9-4601-9450-7a0b2a46e975 None admin] RBAC: Authorizing `identity:get_user(user_id=83b8dc3f532542ddbe8018d329327c24)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:45.135690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95076d4-f2a9-4601-9450-7a0b2a46e975 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:45.137393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 88/189] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:44 2022] GET /identity/v3/users/83b8dc3f532542ddbe8018d329327c24 => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:45.140480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f2cad-0d78-4947-af30-fd2ff2b078df None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:45.140971 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f2cad-0d78-4947-af30-fd2ff2b078df None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:45.141460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f2cad-0d78-4947-af30-fd2ff2b078df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:45.162504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9e1f2cad-0d78-4947-af30-fd2ff2b078df None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:45.163712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9e1f2cad-0d78-4947-af30-fd2ff2b078df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:45.200014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 102/190] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:45.200603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee84ced-3e79-45c8-8ab1-11562a5489d1 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:45.249243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 89/191] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:45.250327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f2cad-0d78-4947-af30-fd2ff2b078df None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:45.348085 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee84ced-3e79-45c8-8ab1-11562a5489d1 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:45.365615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:45.392100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f2cad-0d78-4947-af30-fd2ff2b078df None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:45.396575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d08cb2-a22b-4292-a612-938addbb5584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:45.397389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d08cb2-a22b-4292-a612-938addbb5584 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:45.397668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d08cb2-a22b-4292-a612-938addbb5584 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:45.397944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d08cb2-a22b-4292-a612-938addbb5584 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:45.409386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d08cb2-a22b-4292-a612-938addbb5584 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-46d08cb2-a22b-4292-a612-938addbb5584 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:45.417847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:45.424534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 103/192] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:11:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:45.425619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d08cb2-a22b-4292-a612-938addbb5584 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:45.426391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:45.437999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26289e8d-c663-4b84-98db-b36429e14317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:45.438688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26289e8d-c663-4b84-98db-b36429e14317 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:45.438972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26289e8d-c663-4b84-98db-b36429e14317 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:45.439249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26289e8d-c663-4b84-98db-b36429e14317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:45.519397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26289e8d-c663-4b84-98db-b36429e14317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:45.520858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26289e8d-c663-4b84-98db-b36429e14317 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:45.539186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d08cb2-a22b-4292-a612-938addbb5584 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:45.575345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b398b-e943-4821-8fa4-79733354ad15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:45.576160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b398b-e943-4821-8fa4-79733354ad15 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:45.576446 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b398b-e943-4821-8fa4-79733354ad15 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:45.576725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b398b-e943-4821-8fa4-79733354ad15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:45.578693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b398b-e943-4821-8fa4-79733354ad15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:45.580145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b398b-e943-4821-8fa4-79733354ad15 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:45.581782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 104/193] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:45.582584 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b398b-e943-4821-8fa4-79733354ad15 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:45.617700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:45.649292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 90/194] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:45.657205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26289e8d-c663-4b84-98db-b36429e14317 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:45.679560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:45.719641 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b398b-e943-4821-8fa4-79733354ad15 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:45.772126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a585e-0efd-4df2-b224-abc7db82c669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:45.772939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a585e-0efd-4df2-b224-abc7db82c669 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:45.773215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a585e-0efd-4df2-b224-abc7db82c669 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:45.773496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a585e-0efd-4df2-b224-abc7db82c669 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:45.784928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a585e-0efd-4df2-b224-abc7db82c669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-0c6a585e-0efd-4df2-b224-abc7db82c669 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:45.792841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:45.793718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:45.794505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 105/195] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:45.795319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a585e-0efd-4df2-b224-abc7db82c669 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:45.804028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26289e8d-c663-4b84-98db-b36429e14317 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:45.849684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4ee12-1d50-444f-8eca-85e320b23f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:45.850860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ee12-1d50-444f-8eca-85e320b23f1f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:45.851356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ee12-1d50-444f-8eca-85e320b23f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:45.851714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ee12-1d50-444f-8eca-85e320b23f1f None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:45.852899 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4ee12-1d50-444f-8eca-85e320b23f1f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:45.853423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4ee12-1d50-444f-8eca-85e320b23f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:45.855910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4ee12-1d50-444f-8eca-85e320b23f1f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:45.869861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 91/196] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:11:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:45.872002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4ee12-1d50-444f-8eca-85e320b23f1f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:45.919745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:45.939492 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a585e-0efd-4df2-b224-abc7db82c669 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:45.975133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:45.981885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066ebe0-2748-43cf-bbce-69b523888e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:45.982679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066ebe0-2748-43cf-bbce-69b523888e0b None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:45.983140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066ebe0-2748-43cf-bbce-69b523888e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:45.983701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9066ebe0-2748-43cf-bbce-69b523888e0b None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:45.984628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066ebe0-2748-43cf-bbce-69b523888e0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:45.986454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9066ebe0-2748-43cf-bbce-69b523888e0b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:46.001015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4ee12-1d50-444f-8eca-85e320b23f1f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:46.021551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 106/197] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:11:45 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:46.022843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066ebe0-2748-43cf-bbce-69b523888e0b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:46.066699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d454c12-dfd1-495f-9487-e60c9c3d982b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:46.068060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454c12-dfd1-495f-9487-e60c9c3d982b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:46.068661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454c12-dfd1-495f-9487-e60c9c3d982b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:46.069081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d454c12-dfd1-495f-9487-e60c9c3d982b None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:46.071327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d454c12-dfd1-495f-9487-e60c9c3d982b None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:46.074008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d454c12-dfd1-495f-9487-e60c9c3d982b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:46.077860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 92/198] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:45 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:46.080202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d454c12-dfd1-495f-9487-e60c9c3d982b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:46.208152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9066ebe0-2748-43cf-bbce-69b523888e0b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:46.251184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d454c12-dfd1-495f-9487-e60c9c3d982b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:46.257936 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e020813-bee5-4a20-b1ed-a0eb85b6ee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:46.258653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e020813-bee5-4a20-b1ed-a0eb85b6ee33 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:46.258937 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e020813-bee5-4a20-b1ed-a0eb85b6ee33 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:46.259242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e020813-bee5-4a20-b1ed-a0eb85b6ee33 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:46.260091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e020813-bee5-4a20-b1ed-a0eb85b6ee33 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:46.260398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e020813-bee5-4a20-b1ed-a0eb85b6ee33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:46.262019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e020813-bee5-4a20-b1ed-a0eb85b6ee33 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:46.270842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:46.277082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 107/199] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:46.280090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e020813-bee5-4a20-b1ed-a0eb85b6ee33 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:46.312642 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4cae0-1d77-44b6-8bbc-2cf0b2e10c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:46.314174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4cae0-1d77-44b6-8bbc-2cf0b2e10c7e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:46.314762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4cae0-1d77-44b6-8bbc-2cf0b2e10c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:46.315236 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4cae0-1d77-44b6-8bbc-2cf0b2e10c7e None admin] PATH_INFO: `/v3/users/83b8dc3f532542ddbe8018d329327c24` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:46.317536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb4cae0-1d77-44b6-8bbc-2cf0b2e10c7e None admin] RBAC: Authorizing `identity:get_user(user_id=83b8dc3f532542ddbe8018d329327c24)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:46.320311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb4cae0-1d77-44b6-8bbc-2cf0b2e10c7e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:46.323462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 93/200] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:46 2022] GET /identity/v3/users/83b8dc3f532542ddbe8018d329327c24 => generated 283 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:46.330127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4cae0-1d77-44b6-8bbc-2cf0b2e10c7e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:46.362701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:46.484358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e020813-bee5-4a20-b1ed-a0eb85b6ee33 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:46.532177 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4cae0-1d77-44b6-8bbc-2cf0b2e10c7e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:46.558948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fc01f-2d1c-4a35-bd4e-7745446334b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:46.559619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fc01f-2d1c-4a35-bd4e-7745446334b9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:46.559899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fc01f-2d1c-4a35-bd4e-7745446334b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:46.560174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fc01f-2d1c-4a35-bd4e-7745446334b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:46.562096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fc01f-2d1c-4a35-bd4e-7745446334b9 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user.id=83b8dc3f532542ddbe8018d329327c24) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:46.562413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fc01f-2d1c-4a35-bd4e-7745446334b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:46.564218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fc01f-2d1c-4a35-bd4e-7745446334b9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:46.571146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 108/201] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:46 2022] GET /identity/v3/role_assignments?user.id=83b8dc3f532542ddbe8018d329327c24&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=d67707782358417ba9f25b2793ca2890 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:46.575838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fc01f-2d1c-4a35-bd4e-7745446334b9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:46.638921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58729e3-6348-4981-aa7e-c32f6e9318f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:46.641251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58729e3-6348-4981-aa7e-c32f6e9318f5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:46.642016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58729e3-6348-4981-aa7e-c32f6e9318f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:46.642704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58729e3-6348-4981-aa7e-c32f6e9318f5 None admin] PATH_INFO: `/v3/users/e016498b552f4919a357ac5ec6defa37` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:46.644861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58729e3-6348-4981-aa7e-c32f6e9318f5 None admin] RBAC: Authorizing `identity:get_user(user_id=e016498b552f4919a357ac5ec6defa37)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:46.648309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58729e3-6348-4981-aa7e-c32f6e9318f5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:46.652415 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 94/202] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:46 2022] GET /identity/v3/users/e016498b552f4919a357ac5ec6defa37 => generated 289 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:46.661413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58729e3-6348-4981-aa7e-c32f6e9318f5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:46.839098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:46.910097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58729e3-6348-4981-aa7e-c32f6e9318f5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:46.940647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fc01f-2d1c-4a35-bd4e-7745446334b9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:46.969941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:46.995114 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301e408-dc25-4282-ab1b-eb32de236210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:46.995790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301e408-dc25-4282-ab1b-eb32de236210 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:46.996068 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301e408-dc25-4282-ab1b-eb32de236210 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:46.996396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7301e408-dc25-4282-ab1b-eb32de236210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:47.019762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64be53b-2049-4262-a499-e1f3c8f1b9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:47.020904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64be53b-2049-4262-a499-e1f3c8f1b9be None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:47.021384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64be53b-2049-4262-a499-e1f3c8f1b9be None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:47.021857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64be53b-2049-4262-a499-e1f3c8f1b9be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:47.191201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301e408-dc25-4282-ab1b-eb32de236210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:47.194118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7301e408-dc25-4282-ab1b-eb32de236210 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:47.238733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64be53b-2049-4262-a499-e1f3c8f1b9be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:47.240485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64be53b-2049-4262-a499-e1f3c8f1b9be None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:47.289544 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:47.400899 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 95/203] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 740 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:47.409650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301e408-dc25-4282-ab1b-eb32de236210 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:47.475032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 109/204] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 903 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:47.480879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64be53b-2049-4262-a499-e1f3c8f1b9be None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:47.507133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:47.630576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7301e408-dc25-4282-ab1b-eb32de236210 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:47.715270 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40274e04-c8e7-4c88-83a0-44b4e7bfb88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:47.716601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40274e04-c8e7-4c88-83a0-44b4e7bfb88b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:47.717094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40274e04-c8e7-4c88-83a0-44b4e7bfb88b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:47.718337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40274e04-c8e7-4c88-83a0-44b4e7bfb88b None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:47.721015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40274e04-c8e7-4c88-83a0-44b4e7bfb88b None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:47.722617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40274e04-c8e7-4c88-83a0-44b4e7bfb88b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:47.725946 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 96/205] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:47 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:47.728827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64be53b-2049-4262-a499-e1f3c8f1b9be None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:47.744548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40274e04-c8e7-4c88-83a0-44b4e7bfb88b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:47.778584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:47.831107 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a3b5c-f009-4634-a5c8-25dc2f43ad64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:47.831983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a3b5c-f009-4634-a5c8-25dc2f43ad64 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:47.832258 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a3b5c-f009-4634-a5c8-25dc2f43ad64 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:47.840059 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a3b5c-f009-4634-a5c8-25dc2f43ad64 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:47.844668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a3b5c-f009-4634-a5c8-25dc2f43ad64 None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:47.846160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a3b5c-f009-4634-a5c8-25dc2f43ad64 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:47.852214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 110/206] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:47 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:47.859527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a3b5c-f009-4634-a5c8-25dc2f43ad64 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:47.941300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40274e04-c8e7-4c88-83a0-44b4e7bfb88b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:48.014773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7370d31-1b10-49b9-898d-a2d2f86b95b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:48.017481 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7370d31-1b10-49b9-898d-a2d2f86b95b3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:48.017762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7370d31-1b10-49b9-898d-a2d2f86b95b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:48.018041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7370d31-1b10-49b9-898d-a2d2f86b95b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:48.018919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7370d31-1b10-49b9-898d-a2d2f86b95b3 None admin] RBAC: Adding query filter params (role.id=989320df8c4149cf918861e3ee5954d2, scope.project.id=7139ad672c734cf799e4c39623384e9c, user.id=e016498b552f4919a357ac5ec6defa37) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:48.019231 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7370d31-1b10-49b9-898d-a2d2f86b95b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:48.020720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7370d31-1b10-49b9-898d-a2d2f86b95b3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:48.033211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:48.037465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 97/207] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:47 2022] GET /identity/v3/role_assignments?user.id=e016498b552f4919a357ac5ec6defa37&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=989320df8c4149cf918861e3ee5954d2 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:48.070377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a3b5c-f009-4634-a5c8-25dc2f43ad64 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:48.132621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3f16c-1f81-4c80-b3d7-89d174744a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:48.133761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3f16c-1f81-4c80-b3d7-89d174744a73 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:48.134238 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3f16c-1f81-4c80-b3d7-89d174744a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:48.134607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3f16c-1f81-4c80-b3d7-89d174744a73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:48.136094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3f16c-1f81-4c80-b3d7-89d174744a73 None admin] RBAC: Adding query filter params (role.id=f4461b279e86422b83e29521e7ca29d0, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user.id=83b8dc3f532542ddbe8018d329327c24) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:48.136710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3f16c-1f81-4c80-b3d7-89d174744a73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:48.139431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3f16c-1f81-4c80-b3d7-89d174744a73 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:48.150908 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 111/208] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:47 2022] GET /identity/v3/role_assignments?user.id=83b8dc3f532542ddbe8018d329327c24&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=f4461b279e86422b83e29521e7ca29d0 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:49.621427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ecf6a-ab58-4b26-99e7-41799858aff9 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:49.622193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ecf6a-ab58-4b26-99e7-41799858aff9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:49.622886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ecf6a-ab58-4b26-99e7-41799858aff9 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:49.626018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 98/209] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:49.635730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6492b8f-a646-4535-bb9a-663fda8c4d7b None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:49.636387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6492b8f-a646-4535-bb9a-663fda8c4d7b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:49.636941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6492b8f-a646-4535-bb9a-663fda8c4d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:49.662749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d6492b8f-a646-4535-bb9a-663fda8c4d7b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:49.663972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d6492b8f-a646-4535-bb9a-663fda8c4d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:49.748448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 112/210] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:49.775935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ecf6a-ab58-4b26-99e7-41799858aff9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:49.900928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:49.939423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ecf6a-ab58-4b26-99e7-41799858aff9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:49.960089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7177-f020-41fd-8875-858caaac573d None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:49.960403 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7177-f020-41fd-8875-858caaac573d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:49.960653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7177-f020-41fd-8875-858caaac573d None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:49.962226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 113/211] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:49.968449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fba18-84ea-4f6b-b1d6-1477fe0e7aa2 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:49.968943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fba18-84ea-4f6b-b1d6-1477fe0e7aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:49.969285 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fba18-84ea-4f6b-b1d6-1477fe0e7aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:49.994126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-b34fba18-84ea-4f6b-b1d6-1477fe0e7aa2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:49.995694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b34fba18-84ea-4f6b-b1d6-1477fe0e7aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:50.001155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f7897-41e2-4da6-be41-ea969da982f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:50.001972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f7897-41e2-4da6-be41-ea969da982f2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:50.002259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f7897-41e2-4da6-be41-ea969da982f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:50.002559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f7897-41e2-4da6-be41-ea969da982f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:50.015176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f7897-41e2-4da6-be41-ea969da982f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-5a7f7897-41e2-4da6-be41-ea969da982f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:50.023388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:50.025022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:50.026966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 99/212] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:50.036014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f7897-41e2-4da6-be41-ea969da982f2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:50.108306 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 114/213] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:50.138018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e013c-3206-40a4-b8a8-a25169f252fe None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:50.138465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e013c-3206-40a4-b8a8-a25169f252fe None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:50.138760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e013c-3206-40a4-b8a8-a25169f252fe None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:50.140689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 115/214] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:11:50.146385 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82637f97-43f7-4398-b5d8-030b5c199826 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:50.146706 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82637f97-43f7-4398-b5d8-030b5c199826 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:50.147058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82637f97-43f7-4398-b5d8-030b5c199826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:50.163910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:50.172683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-82637f97-43f7-4398-b5d8-030b5c199826 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:50.173597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-82637f97-43f7-4398-b5d8-030b5c199826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:50.202247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f7897-41e2-4da6-be41-ea969da982f2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:50.262248 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98318f45-7f40-4e58-bd63-b266d2a4cc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:50.263353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98318f45-7f40-4e58-bd63-b266d2a4cc28 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:50.264612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98318f45-7f40-4e58-bd63-b266d2a4cc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:50.265949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98318f45-7f40-4e58-bd63-b266d2a4cc28 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:50.265949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98318f45-7f40-4e58-bd63-b266d2a4cc28 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:50.266523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98318f45-7f40-4e58-bd63-b266d2a4cc28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:50.267746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98318f45-7f40-4e58-bd63-b266d2a4cc28 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:50.276030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:50.285562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 100/215] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:50.287074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98318f45-7f40-4e58-bd63-b266d2a4cc28 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:50.304770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 116/216] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:50.306643 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82637f97-43f7-4398-b5d8-030b5c199826 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:50.468200 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98318f45-7f40-4e58-bd63-b266d2a4cc28 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:50.488296 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82637f97-43f7-4398-b5d8-030b5c199826 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:50.523507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d0459-ae93-4e44-9e65-3fb5a6a6b814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:50.524328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d0459-ae93-4e44-9e65-3fb5a6a6b814 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:50.524610 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d0459-ae93-4e44-9e65-3fb5a6a6b814 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:50.524891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d0459-ae93-4e44-9e65-3fb5a6a6b814 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:50.529587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1402323-8517-4487-b648-2c38240282d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:50.530567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402323-8517-4487-b648-2c38240282d6 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:50.530837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402323-8517-4487-b648-2c38240282d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:50.531116 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1402323-8517-4487-b648-2c38240282d6 None admin] PATH_INFO: `/v3/users/97bd60be6f5a4095b17d73438c427592` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:50.534354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:50.543932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d0459-ae93-4e44-9e65-3fb5a6a6b814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:50.549567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1402323-8517-4487-b648-2c38240282d6 None admin] RBAC: Authorizing `identity:get_user(user_id=97bd60be6f5a4095b17d73438c427592)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:50.551765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1402323-8517-4487-b648-2c38240282d6 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-5e8d0459-ae93-4e44-9e65-3fb5a6a6b814 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:50.552057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:50.553566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 101/217] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:11:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:50.556221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d0459-ae93-4e44-9e65-3fb5a6a6b814 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:50.560516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 117/218] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:50 2022] GET /identity/v3/users/97bd60be6f5a4095b17d73438c427592 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:50.562491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1402323-8517-4487-b648-2c38240282d6 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:50.658969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:50.727077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1402323-8517-4487-b648-2c38240282d6 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:50.732328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d0459-ae93-4e44-9e65-3fb5a6a6b814 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:50.768946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c735ce-15be-47d7-97a0-7dd86a4587f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:50.770328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c735ce-15be-47d7-97a0-7dd86a4587f2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:50.771025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c735ce-15be-47d7-97a0-7dd86a4587f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:50.771526 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c735ce-15be-47d7-97a0-7dd86a4587f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:50.772722 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c735ce-15be-47d7-97a0-7dd86a4587f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:50.773098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c735ce-15be-47d7-97a0-7dd86a4587f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:50.775622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c735ce-15be-47d7-97a0-7dd86a4587f2 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:50.776038 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827465bf-e9b1-4978-85e2-15e681cfcebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:50.777269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827465bf-e9b1-4978-85e2-15e681cfcebd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:50.777766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827465bf-e9b1-4978-85e2-15e681cfcebd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:50.778097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827465bf-e9b1-4978-85e2-15e681cfcebd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:50.779411 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827465bf-e9b1-4978-85e2-15e681cfcebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:50.781075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827465bf-e9b1-4978-85e2-15e681cfcebd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:50.786624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 102/219] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:50.786624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827465bf-e9b1-4978-85e2-15e681cfcebd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:50.795073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 118/220] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:50.797399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c735ce-15be-47d7-97a0-7dd86a4587f2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:50.874583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:50.929517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827465bf-e9b1-4978-85e2-15e681cfcebd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:50.944816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c735ce-15be-47d7-97a0-7dd86a4587f2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:50.979868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:50.991859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466e5e3-7eaa-489a-b5ae-172024fe76a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:50.992516 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466e5e3-7eaa-489a-b5ae-172024fe76a2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:50.992800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466e5e3-7eaa-489a-b5ae-172024fe76a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:50.993071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466e5e3-7eaa-489a-b5ae-172024fe76a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:50.995412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb3406-5860-4564-a9d1-43da7f7dfb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:50.996083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb3406-5860-4564-a9d1-43da7f7dfb0b None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:50.996358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb3406-5860-4564-a9d1-43da7f7dfb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:50.996618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb3406-5860-4564-a9d1-43da7f7dfb0b None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:50.997676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb3406-5860-4564-a9d1-43da7f7dfb0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:50.999466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb3406-5860-4564-a9d1-43da7f7dfb0b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:51.027157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 119/221] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:11:50 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:11:51.028373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb3406-5860-4564-a9d1-43da7f7dfb0b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:51.128198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466e5e3-7eaa-489a-b5ae-172024fe76a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:51.131274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d466e5e3-7eaa-489a-b5ae-172024fe76a2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:51.245649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb3406-5860-4564-a9d1-43da7f7dfb0b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:51.281433 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:51.328818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 103/222] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:51.336199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466e5e3-7eaa-489a-b5ae-172024fe76a2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:51.340622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf171d-5cf9-42fc-87e6-6f30b9a1042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:51.341546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf171d-5cf9-42fc-87e6-6f30b9a1042a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:51.341998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf171d-5cf9-42fc-87e6-6f30b9a1042a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:51.342843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf171d-5cf9-42fc-87e6-6f30b9a1042a None admin] PATH_INFO: `/v3/users/83b8dc3f532542ddbe8018d329327c24` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:51.345089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf171d-5cf9-42fc-87e6-6f30b9a1042a None admin] RBAC: Authorizing `identity:get_user(user_id=83b8dc3f532542ddbe8018d329327c24)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:51.346654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf171d-5cf9-42fc-87e6-6f30b9a1042a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:51.348694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 120/223] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:51 2022] GET /identity/v3/users/83b8dc3f532542ddbe8018d329327c24 => generated 283 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:51.354166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf171d-5cf9-42fc-87e6-6f30b9a1042a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:51.398271 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:51.552051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf171d-5cf9-42fc-87e6-6f30b9a1042a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:51.563992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466e5e3-7eaa-489a-b5ae-172024fe76a2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:51.621650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2c3f5-da86-4486-9685-37ef3a6a3547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:51.623178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2c3f5-da86-4486-9685-37ef3a6a3547 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:51.623662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2c3f5-da86-4486-9685-37ef3a6a3547 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:51.624187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2c3f5-da86-4486-9685-37ef3a6a3547 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:51.626305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2c3f5-da86-4486-9685-37ef3a6a3547 None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:51.629091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2c3f5-da86-4486-9685-37ef3a6a3547 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:51.632715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 121/224] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:51 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:51.635688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff084aef-f0ca-47c8-a552-7d0fc8931f03 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:51.636129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff084aef-f0ca-47c8-a552-7d0fc8931f03 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:51.636298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a9ba2-a372-4ff7-8346-0cfcb2a1ef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:51.636613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff084aef-f0ca-47c8-a552-7d0fc8931f03 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:51.637618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a9ba2-a372-4ff7-8346-0cfcb2a1ef16 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:51.638075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a9ba2-a372-4ff7-8346-0cfcb2a1ef16 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:51.638522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a9ba2-a372-4ff7-8346-0cfcb2a1ef16 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:51.639526 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 122/225] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:51.642571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a9ba2-a372-4ff7-8346-0cfcb2a1ef16 None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:51.643346 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff084aef-f0ca-47c8-a552-7d0fc8931f03 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:51.646173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a9ba2-a372-4ff7-8346-0cfcb2a1ef16 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:51.649648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 104/226] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:51 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:51.656123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec165452-4be6-4e67-97e8-a9b5d2b69bf2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:51.656580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec165452-4be6-4e67-97e8-a9b5d2b69bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:51.657055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec165452-4be6-4e67-97e8-a9b5d2b69bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:51.684620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ec165452-4be6-4e67-97e8-a9b5d2b69bf2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:51.685845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ec165452-4be6-4e67-97e8-a9b5d2b69bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:51.750593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:51.828001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff084aef-f0ca-47c8-a552-7d0fc8931f03 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:51.833267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 105/227] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:51.833981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec165452-4be6-4e67-97e8-a9b5d2b69bf2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:51.868529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:51.892612 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743940f5-5ad0-4b1d-b708-585589d7eb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:51.893520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743940f5-5ad0-4b1d-b708-585589d7eb9e None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:51.893798 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743940f5-5ad0-4b1d-b708-585589d7eb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:51.894078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743940f5-5ad0-4b1d-b708-585589d7eb9e None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/users/83b8dc3f532542ddbe8018d329327c24/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:51.896253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743940f5-5ad0-4b1d-b708-585589d7eb9e None admin] RBAC: Authorizing `identity:create_grant(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user_id=83b8dc3f532542ddbe8018d329327c24, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:51.897731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743940f5-5ad0-4b1d-b708-585589d7eb9e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:51.915730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 123/228] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:11:51 2022] PUT /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/users/83b8dc3f532542ddbe8018d329327c24/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:11:51.916608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743940f5-5ad0-4b1d-b708-585589d7eb9e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:52.008207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec165452-4be6-4e67-97e8-a9b5d2b69bf2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:52.050567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031455c-bd64-4059-9c77-44f03741a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:52.051337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031455c-bd64-4059-9c77-44f03741a9cf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:52.051689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031455c-bd64-4059-9c77-44f03741a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:52.051975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031455c-bd64-4059-9c77-44f03741a9cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:52.052839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031455c-bd64-4059-9c77-44f03741a9cf None admin] RBAC: Adding query filter params (role.id=1249af5cf5054410a11b2d88c03fcd52, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user.id=97bd60be6f5a4095b17d73438c427592) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:52.053308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031455c-bd64-4059-9c77-44f03741a9cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:52.054853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5031455c-bd64-4059-9c77-44f03741a9cf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:52.061945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 106/229] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:51 2022] GET /identity/v3/role_assignments?user.id=97bd60be6f5a4095b17d73438c427592&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:52.088210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743940f5-5ad0-4b1d-b708-585589d7eb9e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:52.100048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:52.131316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f89d5-47ff-4444-a66a-6da8705f323b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:52.132164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f89d5-47ff-4444-a66a-6da8705f323b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:52.132446 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f89d5-47ff-4444-a66a-6da8705f323b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:52.132725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f89d5-47ff-4444-a66a-6da8705f323b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:52.146464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f89d5-47ff-4444-a66a-6da8705f323b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-252f89d5-47ff-4444-a66a-6da8705f323b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:52.155105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:52.157596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:52.159246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 124/230] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:11:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:52.163741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031455c-bd64-4059-9c77-44f03741a9cf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:52.317493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:52.329796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031455c-bd64-4059-9c77-44f03741a9cf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:52.369697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60887794-3884-4744-8164-57f120728e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:52.370294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60887794-3884-4744-8164-57f120728e84 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:52.370607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60887794-3884-4744-8164-57f120728e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:52.370887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60887794-3884-4744-8164-57f120728e84 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:52.371663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60887794-3884-4744-8164-57f120728e84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:52.371968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60887794-3884-4744-8164-57f120728e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:52.373465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60887794-3884-4744-8164-57f120728e84 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:52.383720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 107/231] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:11:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:52.389483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f89d5-47ff-4444-a66a-6da8705f323b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:52.549416 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f89d5-47ff-4444-a66a-6da8705f323b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:52.599385 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c744b3-eb5a-4b8a-a196-fb9b1d775f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:52.600265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c744b3-eb5a-4b8a-a196-fb9b1d775f88 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:52.600625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c744b3-eb5a-4b8a-a196-fb9b1d775f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:52.600956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c744b3-eb5a-4b8a-a196-fb9b1d775f88 None admin] PATH_INFO: `/v3/users/83b8dc3f532542ddbe8018d329327c24` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:52.602839 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c744b3-eb5a-4b8a-a196-fb9b1d775f88 None admin] RBAC: Authorizing `identity:get_user(user_id=83b8dc3f532542ddbe8018d329327c24)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:52.605018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c744b3-eb5a-4b8a-a196-fb9b1d775f88 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:52.609775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 125/232] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:52 2022] GET /identity/v3/users/83b8dc3f532542ddbe8018d329327c24 => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:52.617207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60887794-3884-4744-8164-57f120728e84 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:52.760653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60887794-3884-4744-8164-57f120728e84 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:52.815421 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a119e4a-c707-4818-8012-3514e8a03e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:52.816718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119e4a-c707-4818-8012-3514e8a03e0f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:52.817130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119e4a-c707-4818-8012-3514e8a03e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:52.817521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a119e4a-c707-4818-8012-3514e8a03e0f None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:52.819219 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a119e4a-c707-4818-8012-3514e8a03e0f None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:52.821403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a119e4a-c707-4818-8012-3514e8a03e0f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:52.824363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 108/233] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:52 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:52.832559 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c744b3-eb5a-4b8a-a196-fb9b1d775f88 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:52.892950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:53.011631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c744b3-eb5a-4b8a-a196-fb9b1d775f88 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:53.050271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef380b-60ce-4ec3-bdb5-a31746577e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:53.051192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef380b-60ce-4ec3-bdb5-a31746577e32 None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:53.051520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef380b-60ce-4ec3-bdb5-a31746577e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:53.051802 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef380b-60ce-4ec3-bdb5-a31746577e32 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/users/83b8dc3f532542ddbe8018d329327c24/roles/f4461b279e86422b83e29521e7ca29d0` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:53.054076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef380b-60ce-4ec3-bdb5-a31746577e32 None admin] RBAC: Authorizing `identity:create_grant(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user_id=83b8dc3f532542ddbe8018d329327c24, role_id=f4461b279e86422b83e29521e7ca29d0)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:53.055573 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef380b-60ce-4ec3-bdb5-a31746577e32 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:53.072341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 126/234] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:11:52 2022] PUT /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/users/83b8dc3f532542ddbe8018d329327c24/roles/f4461b279e86422b83e29521e7ca29d0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:11:54.616551 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa5f51-380b-4ebd-a71d-0d9d7a1210b0 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:54.617536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa5f51-380b-4ebd-a71d-0d9d7a1210b0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:54.618080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa5f51-380b-4ebd-a71d-0d9d7a1210b0 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:54.620635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 109/235] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:54.623452 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906df056-543a-42d1-bd9c-4ba0ce367100 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:54.624749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906df056-543a-42d1-bd9c-4ba0ce367100 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:54.625483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906df056-543a-42d1-bd9c-4ba0ce367100 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:54.628240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 127/236] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:54.635669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a951c-9e1d-45d3-b810-03cb0145fe31 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:54.636522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a951c-9e1d-45d3-b810-03cb0145fe31 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:54.637264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a951c-9e1d-45d3-b810-03cb0145fe31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:54.649017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e08fa5-454c-4e01-abe3-d34c5664f230 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:54.649657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e08fa5-454c-4e01-abe3-d34c5664f230 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:54.655930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e08fa5-454c-4e01-abe3-d34c5664f230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:54.680046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-089a951c-9e1d-45d3-b810-03cb0145fe31 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:54.681522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-089a951c-9e1d-45d3-b810-03cb0145fe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:54.696281 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-41e08fa5-454c-4e01-abe3-d34c5664f230 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:54.697497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-41e08fa5-454c-4e01-abe3-d34c5664f230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:54.708529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:54.787596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 110/237] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:54.837822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 128/238] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:54.844342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a951c-9e1d-45d3-b810-03cb0145fe31 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:54.918160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e08fa5-454c-4e01-abe3-d34c5664f230 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:54.988763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:55.004677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a951c-9e1d-45d3-b810-03cb0145fe31 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:55.056790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66644eff-781e-4db3-ba70-bb3d289f52ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:55.058078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66644eff-781e-4db3-ba70-bb3d289f52ea None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:55.058460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66644eff-781e-4db3-ba70-bb3d289f52ea None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:55.058897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66644eff-781e-4db3-ba70-bb3d289f52ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:55.060322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66644eff-781e-4db3-ba70-bb3d289f52ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:55.062133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66644eff-781e-4db3-ba70-bb3d289f52ea None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:55.065263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 111/239] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:11:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:55.072046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66644eff-781e-4db3-ba70-bb3d289f52ea None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:55.098781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:55.206479 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e08fa5-454c-4e01-abe3-d34c5664f230 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:55.249456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66644eff-781e-4db3-ba70-bb3d289f52ea None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:55.289287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d116fc-d54c-4074-abd3-47fb939976ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:55.290327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d116fc-d54c-4074-abd3-47fb939976ed None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:55.290658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d116fc-d54c-4074-abd3-47fb939976ed None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:55.290939 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d116fc-d54c-4074-abd3-47fb939976ed None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:55.291689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d116fc-d54c-4074-abd3-47fb939976ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:55.293183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d116fc-d54c-4074-abd3-47fb939976ed None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:55.317269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 112/240] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:11:55 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:11:55.328517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50c96-0601-42bd-96eb-f149b98f9936 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:55.328797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50c96-0601-42bd-96eb-f149b98f9936 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:55.329129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f50c96-0601-42bd-96eb-f149b98f9936 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:55.330513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 113/241] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:11:55.331554 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56d390-1c02-4948-921c-62631ea1babe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:55.332763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56d390-1c02-4948-921c-62631ea1babe None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:55.333270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56d390-1c02-4948-921c-62631ea1babe None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:55.333775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56d390-1c02-4948-921c-62631ea1babe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:55.344664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4efd8-8fbf-4771-9995-540f0777095c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:55.345182 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4efd8-8fbf-4771-9995-540f0777095c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:55.345714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4efd8-8fbf-4771-9995-540f0777095c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:55.372646 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56d390-1c02-4948-921c-62631ea1babe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:55.376694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c9a4efd8-8fbf-4771-9995-540f0777095c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:55.378228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c9a4efd8-8fbf-4771-9995-540f0777095c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ae56d390-1c02-4948-921c-62631ea1babe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:55.385683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:55.389861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:11:55.392760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:55.402194 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 129/242] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 477 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:11:55.402194 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56d390-1c02-4948-921c-62631ea1babe None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:55.510497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 114/243] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:55.536059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4efd8-8fbf-4771-9995-540f0777095c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:55.541431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:55.664163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:55.698524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56d390-1c02-4948-921c-62631ea1babe None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:55.732182 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4efd8-8fbf-4771-9995-540f0777095c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:55.787242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b172264-4d5b-4f07-9db5-ee6f7365a96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:55.787846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b172264-4d5b-4f07-9db5-ee6f7365a96d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:55.788158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b172264-4d5b-4f07-9db5-ee6f7365a96d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:55.788478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b172264-4d5b-4f07-9db5-ee6f7365a96d None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:55.789326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b172264-4d5b-4f07-9db5-ee6f7365a96d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:55.790315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b172264-4d5b-4f07-9db5-ee6f7365a96d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:55.791441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b172264-4d5b-4f07-9db5-ee6f7365a96d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:55.809022 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 130/244] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:55.817219 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b172264-4d5b-4f07-9db5-ee6f7365a96d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:55.818203 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85cca97-a125-4e5b-8c8a-06094015b9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:55.819089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cca97-a125-4e5b-8c8a-06094015b9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:55.819378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cca97-a125-4e5b-8c8a-06094015b9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:55.819683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cca97-a125-4e5b-8c8a-06094015b9b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:55.836638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85cca97-a125-4e5b-8c8a-06094015b9b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c85cca97-a125-4e5b-8c8a-06094015b9b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:55.846555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:55.847514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:11:55.848641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:55.849661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 115/245] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:55.855604 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b05306-119c-463e-8528-d486ba2328a3 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:55.856260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b05306-119c-463e-8528-d486ba2328a3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:55.856775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b05306-119c-463e-8528-d486ba2328a3 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:55.859392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 116/246] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:55.860797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b05306-119c-463e-8528-d486ba2328a3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:55.954689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b172264-4d5b-4f07-9db5-ee6f7365a96d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:55.995930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77ef2e-a5d4-46e3-aecf-ed28ed050448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:55.997506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ef2e-a5d4-46e3-aecf-ed28ed050448 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:55.998051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ef2e-a5d4-46e3-aecf-ed28ed050448 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:55.999200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ef2e-a5d4-46e3-aecf-ed28ed050448 None admin] PATH_INFO: `/v3/users/f2721a47fd464e4fa5ff08a701c1dede` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:56.006133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:56.018125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77ef2e-a5d4-46e3-aecf-ed28ed050448 None admin] RBAC: Authorizing `identity:get_user(user_id=f2721a47fd464e4fa5ff08a701c1dede)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:56.019953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77ef2e-a5d4-46e3-aecf-ed28ed050448 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:56.022420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 131/247] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:55 2022] GET /identity/v3/users/f2721a47fd464e4fa5ff08a701c1dede => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:56.029556 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72bfc-dfff-480c-8fa6-505d5bb39182 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:56.029991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72bfc-dfff-480c-8fa6-505d5bb39182 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:56.030391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d72bfc-dfff-480c-8fa6-505d5bb39182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:56.047451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:56.054172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-89d72bfc-dfff-480c-8fa6-505d5bb39182 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:56.055831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-89d72bfc-dfff-480c-8fa6-505d5bb39182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:56.102828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b05306-119c-463e-8528-d486ba2328a3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:56.159316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 132/248] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:56.166829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72bfc-dfff-480c-8fa6-505d5bb39182 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:56.167722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf4743-bae6-4c35-ba5a-d7c05eb7ea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:56.168611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4743-bae6-4c35-ba5a-d7c05eb7ea8f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:56.169122 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4743-bae6-4c35-ba5a-d7c05eb7ea8f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:56.169537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4743-bae6-4c35-ba5a-d7c05eb7ea8f None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:56.171423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf4743-bae6-4c35-ba5a-d7c05eb7ea8f None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:56.171898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf4743-bae6-4c35-ba5a-d7c05eb7ea8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:56.174606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf4743-bae6-4c35-ba5a-d7c05eb7ea8f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:56.185925 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 117/249] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:11:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:56.192579 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf4743-bae6-4c35-ba5a-d7c05eb7ea8f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:56.325352 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d72bfc-dfff-480c-8fa6-505d5bb39182 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:56.368885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d69612-2864-42bc-80d4-8a256beb39df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:56.370169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d69612-2864-42bc-80d4-8a256beb39df None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:56.370671 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d69612-2864-42bc-80d4-8a256beb39df None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:56.371126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d69612-2864-42bc-80d4-8a256beb39df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:56.375901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:56.464897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d69612-2864-42bc-80d4-8a256beb39df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:56.468299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d69612-2864-42bc-80d4-8a256beb39df None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:56.499387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf4743-bae6-4c35-ba5a-d7c05eb7ea8f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:56.539484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:56.551207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a3884-7c10-4609-9fb5-9474afeb2724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:56.552376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a3884-7c10-4609-9fb5-9474afeb2724 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:56.552664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a3884-7c10-4609-9fb5-9474afeb2724 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:56.552943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a3884-7c10-4609-9fb5-9474afeb2724 None admin] PATH_INFO: `/v3/users/97bd60be6f5a4095b17d73438c427592` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:56.554165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a3884-7c10-4609-9fb5-9474afeb2724 None admin] RBAC: Authorizing `identity:get_user(user_id=97bd60be6f5a4095b17d73438c427592)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:56.555694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9a3884-7c10-4609-9fb5-9474afeb2724 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:56.557591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 118/250] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:56 2022] GET /identity/v3/users/97bd60be6f5a4095b17d73438c427592 => generated 297 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:56.558153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a3884-7c10-4609-9fb5-9474afeb2724 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:56.623137 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 133/251] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:56.624555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d69612-2864-42bc-80d4-8a256beb39df None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:56.704100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:56.737843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9a3884-7c10-4609-9fb5-9474afeb2724 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:56.836958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f7dc4-9b23-47fa-ab5e-94116c920a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:56.838322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f7dc4-9b23-47fa-ab5e-94116c920a48 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:56.838846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f7dc4-9b23-47fa-ab5e-94116c920a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:56.839330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f7dc4-9b23-47fa-ab5e-94116c920a48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:56.877713 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d69612-2864-42bc-80d4-8a256beb39df None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:56.878796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f7dc4-9b23-47fa-ab5e-94116c920a48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f11f7dc4-9b23-47fa-ab5e-94116c920a48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:56.889999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:56.890940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:11:56.891733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:11:56.894120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 119/252] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:11:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:56.898829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f7dc4-9b23-47fa-ab5e-94116c920a48 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:56.929502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeae499-e4a2-480c-8b3c-778bbee23dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:56.930748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeae499-e4a2-480c-8b3c-778bbee23dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:56.931247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeae499-e4a2-480c-8b3c-778bbee23dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:56.931713 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeae499-e4a2-480c-8b3c-778bbee23dc7 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:56.933254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeae499-e4a2-480c-8b3c-778bbee23dc7 None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:56.935149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeae499-e4a2-480c-8b3c-778bbee23dc7 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:56.938018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 134/253] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:56 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:56.939611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff9c14-d9cd-401d-b46d-16923bee234d None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:56.940096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff9c14-d9cd-401d-b46d-16923bee234d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:56.940565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff9c14-d9cd-401d-b46d-16923bee234d None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:56.942043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 135/254] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:56.942977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff9c14-d9cd-401d-b46d-16923bee234d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:57.004050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:57.075184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f7dc4-9b23-47fa-ab5e-94116c920a48 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:57.133028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5c36-bb90-4e3d-8110-19fb7f43b90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:57.134767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5c36-bb90-4e3d-8110-19fb7f43b90b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:57.135574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5c36-bb90-4e3d-8110-19fb7f43b90b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:57.136226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5c36-bb90-4e3d-8110-19fb7f43b90b None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:57.138274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f5c36-bb90-4e3d-8110-19fb7f43b90b None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:57.138587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff9c14-d9cd-401d-b46d-16923bee234d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:57.141188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f5c36-bb90-4e3d-8110-19fb7f43b90b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:57.143273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:57.150807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 120/255] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:56 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:57.152036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5c36-bb90-4e3d-8110-19fb7f43b90b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:57.192593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7ec41-a4ef-4317-b419-b63bd55d8a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:57.193687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7ec41-a4ef-4317-b419-b63bd55d8a54 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:57.194175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7ec41-a4ef-4317-b419-b63bd55d8a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:57.194695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7ec41-a4ef-4317-b419-b63bd55d8a54 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:57.195736 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7ec41-a4ef-4317-b419-b63bd55d8a54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:57.196529 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7ec41-a4ef-4317-b419-b63bd55d8a54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:57.199410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7ec41-a4ef-4317-b419-b63bd55d8a54 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:57.222562 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 136/256] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:57.225509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb0d0-ed03-4dba-bbd3-3baff813e485 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:57.226361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb0d0-ed03-4dba-bbd3-3baff813e485 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:57.227153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8fb0d0-ed03-4dba-bbd3-3baff813e485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:57.256011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-eb8fb0d0-ed03-4dba-bbd3-3baff813e485 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:57.258298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-eb8fb0d0-ed03-4dba-bbd3-3baff813e485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:57.344431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5c36-bb90-4e3d-8110-19fb7f43b90b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:57.373146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 137/257] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:57.376093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fb0d0-ed03-4dba-bbd3-3baff813e485 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:57.395975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:57.422719 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6ee02-4ba5-4e5a-80b0-ed9e153b1b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:57.423633 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6ee02-4ba5-4e5a-80b0-ed9e153b1b24 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:57.423952 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6ee02-4ba5-4e5a-80b0-ed9e153b1b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:57.424218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6ee02-4ba5-4e5a-80b0-ed9e153b1b24 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/users/97bd60be6f5a4095b17d73438c427592/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:57.429521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6ee02-4ba5-4e5a-80b0-ed9e153b1b24 None admin] RBAC: Authorizing `identity:create_grant(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user_id=97bd60be6f5a4095b17d73438c427592, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:57.432385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6ee02-4ba5-4e5a-80b0-ed9e153b1b24 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:57.454279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 121/258] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:11:57 2022] PUT /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/users/97bd60be6f5a4095b17d73438c427592/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:11:57.454279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6ee02-4ba5-4e5a-80b0-ed9e153b1b24 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:57.617485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:57.640547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8fb0d0-ed03-4dba-bbd3-3baff813e485 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:57.704246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6ee02-4ba5-4e5a-80b0-ed9e153b1b24 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:57.757510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb338a1-d3f5-436f-bf7b-421152e5cbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:57.758707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb338a1-d3f5-436f-bf7b-421152e5cbec None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:57.759211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb338a1-d3f5-436f-bf7b-421152e5cbec None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:57.759712 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb338a1-d3f5-436f-bf7b-421152e5cbec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:57.761488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb338a1-d3f5-436f-bf7b-421152e5cbec None admin] RBAC: Adding query filter params (role.id=989320df8c4149cf918861e3ee5954d2, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user.id=f2721a47fd464e4fa5ff08a701c1dede) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:57.762045 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb338a1-d3f5-436f-bf7b-421152e5cbec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:57.766438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb338a1-d3f5-436f-bf7b-421152e5cbec None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:57.775617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 138/259] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:57 2022] GET /identity/v3/role_assignments?user.id=f2721a47fd464e4fa5ff08a701c1dede&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=989320df8c4149cf918861e3ee5954d2 => generated 259 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:57.776725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb338a1-d3f5-436f-bf7b-421152e5cbec None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:57.782843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de67747-afa9-49bb-b96b-79fed1a9f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:57.785530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de67747-afa9-49bb-b96b-79fed1a9f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:57.786326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de67747-afa9-49bb-b96b-79fed1a9f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:57.787081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de67747-afa9-49bb-b96b-79fed1a9f5d4 None admin] PATH_INFO: `/v3/users/83b8dc3f532542ddbe8018d329327c24` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:57.788890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de67747-afa9-49bb-b96b-79fed1a9f5d4 None admin] RBAC: Authorizing `identity:get_user(user_id=83b8dc3f532542ddbe8018d329327c24)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:57.790992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de67747-afa9-49bb-b96b-79fed1a9f5d4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:57.794228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 122/260] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:57 2022] GET /identity/v3/users/83b8dc3f532542ddbe8018d329327c24 => generated 283 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:57.804315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de67747-afa9-49bb-b96b-79fed1a9f5d4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:57.886777 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:58.009898 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb338a1-d3f5-436f-bf7b-421152e5cbec None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:58.035157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de67747-afa9-49bb-b96b-79fed1a9f5d4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:58.118358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5cb50-4cb8-4335-9b6d-97b59a0e7539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:58.119227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5cb50-4cb8-4335-9b6d-97b59a0e7539 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:58.119508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5cb50-4cb8-4335-9b6d-97b59a0e7539 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:58.119785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5cb50-4cb8-4335-9b6d-97b59a0e7539 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:58.122513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a966d6-ea1a-4b96-9913-568b61efd5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:58.123889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a966d6-ea1a-4b96-9913-568b61efd5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:58.124178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a966d6-ea1a-4b96-9913-568b61efd5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:58.124498 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a966d6-ea1a-4b96-9913-568b61efd5d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:58.155332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5cb50-4cb8-4335-9b6d-97b59a0e7539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-94f5cb50-4cb8-4335-9b6d-97b59a0e7539 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:11:58.171132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:11:58.172837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:11:58.174467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 139/261] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:11:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:11:58.180489 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5cb50-4cb8-4335-9b6d-97b59a0e7539 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:58.226187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:58.328406 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a966d6-ea1a-4b96-9913-568b61efd5d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:58.329826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a966d6-ea1a-4b96-9913-568b61efd5d1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:58.455151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:58.470246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5cb50-4cb8-4335-9b6d-97b59a0e7539 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:58.579564 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24213773-c846-465d-943e-7c426e7e1364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:58.580170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24213773-c846-465d-943e-7c426e7e1364 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:58.580458 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24213773-c846-465d-943e-7c426e7e1364 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:58.580737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24213773-c846-465d-943e-7c426e7e1364 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:58.581683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24213773-c846-465d-943e-7c426e7e1364 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:58.582181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24213773-c846-465d-943e-7c426e7e1364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:58.583760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24213773-c846-465d-943e-7c426e7e1364 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:58.587460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 123/262] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 784 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:58.600663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a966d6-ea1a-4b96-9913-568b61efd5d1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:58.604823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 140/263] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:11:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:58.618564 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24213773-c846-465d-943e-7c426e7e1364 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:58.820909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a966d6-ea1a-4b96-9913-568b61efd5d1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:58.907002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:58.910998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a700f-ec2c-4377-9ae3-f47cfe2a48a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:58.912422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a700f-ec2c-4377-9ae3-f47cfe2a48a4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:58.912932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a700f-ec2c-4377-9ae3-f47cfe2a48a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:58.913438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a700f-ec2c-4377-9ae3-f47cfe2a48a4 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:58.915983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a700f-ec2c-4377-9ae3-f47cfe2a48a4 None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:58.918769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a700f-ec2c-4377-9ae3-f47cfe2a48a4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:58.922022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 124/264] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:58 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:58.928711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a700f-ec2c-4377-9ae3-f47cfe2a48a4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:58.963522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24213773-c846-465d-943e-7c426e7e1364 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:58.971520 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:59.075671 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90772b0-37a9-475f-b89a-f34e75c9cf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:59.080084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90772b0-37a9-475f-b89a-f34e75c9cf42 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:59.080084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90772b0-37a9-475f-b89a-f34e75c9cf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:59.080416 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90772b0-37a9-475f-b89a-f34e75c9cf42 None admin] PATH_INFO: `/v3/users/83b8dc3f532542ddbe8018d329327c24` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:59.088180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90772b0-37a9-475f-b89a-f34e75c9cf42 None admin] RBAC: Authorizing `identity:get_user(user_id=83b8dc3f532542ddbe8018d329327c24)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:59.093709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90772b0-37a9-475f-b89a-f34e75c9cf42 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:59.093709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 141/265] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:11:58 2022] GET /identity/v3/users/83b8dc3f532542ddbe8018d329327c24 => generated 283 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:11:59.104408 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90772b0-37a9-475f-b89a-f34e75c9cf42 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:59.134596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a700f-ec2c-4377-9ae3-f47cfe2a48a4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:59.224194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79482b-bb68-416b-a935-e6d7bf80b640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:59.225349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79482b-bb68-416b-a935-e6d7bf80b640 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:59.225839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79482b-bb68-416b-a935-e6d7bf80b640 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:59.226316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79482b-bb68-416b-a935-e6d7bf80b640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:59.227524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79482b-bb68-416b-a935-e6d7bf80b640 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user.id=83b8dc3f532542ddbe8018d329327c24) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:11:59.228073 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79482b-bb68-416b-a935-e6d7bf80b640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:59.229831 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79482b-bb68-416b-a935-e6d7bf80b640 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:59.247377 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 125/266] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:11:58 2022] GET /identity/v3/role_assignments?user.id=83b8dc3f532542ddbe8018d329327c24&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=d67707782358417ba9f25b2793ca2890 => generated 608 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:11:59.365227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90772b0-37a9-475f-b89a-f34e75c9cf42 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:59.440384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08d85d-93e7-4ca6-8d5b-a7e0af7af2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:11:59.441091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08d85d-93e7-4ca6-8d5b-a7e0af7af2db None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:59.441366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08d85d-93e7-4ca6-8d5b-a7e0af7af2db None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:59.441645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08d85d-93e7-4ca6-8d5b-a7e0af7af2db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:59.480882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:59.514293 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887a88b-07f3-4b4f-a8fc-7c4557cebe34 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:59.519246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887a88b-07f3-4b4f-a8fc-7c4557cebe34 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:59.520075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887a88b-07f3-4b4f-a8fc-7c4557cebe34 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:59.523465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 126/267] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:59 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:59.541199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f5019-4433-4385-84eb-6ad6d674d241 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:59.542178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f5019-4433-4385-84eb-6ad6d674d241 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:59.542992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f5019-4433-4385-84eb-6ad6d674d241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:59.586597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08d85d-93e7-4ca6-8d5b-a7e0af7af2db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:11:59.586597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08d85d-93e7-4ca6-8d5b-a7e0af7af2db None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:11:59.605954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-964f5019-4433-4385-84eb-6ad6d674d241 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:59.610269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-964f5019-4433-4385-84eb-6ad6d674d241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:59.777279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 142/268] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:11:59.778781 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454a134-074c-46dd-94b2-43941a5d14aa None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:59.779086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454a134-074c-46dd-94b2-43941a5d14aa None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:59.780290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454a134-074c-46dd-94b2-43941a5d14aa None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:59.786620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 143/269] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:11:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:11:59.787493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454a134-074c-46dd-94b2-43941a5d14aa None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:11:59.792936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 127/270] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:11:59.796965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a42b67-c37c-4ec5-aed9-e019ce24bdea None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:11:59.797501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a42b67-c37c-4ec5-aed9-e019ce24bdea None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:11:59.798016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a42b67-c37c-4ec5-aed9-e019ce24bdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:11:59.871048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e7a42b67-c37c-4ec5-aed9-e019ce24bdea None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:11:59.873079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e7a42b67-c37c-4ec5-aed9-e019ce24bdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:11:59.878458 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:59.890887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:11:59.953213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454a134-074c-46dd-94b2-43941a5d14aa None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:11:59.995545 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 128/271] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:11:59.996679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a42b67-c37c-4ec5-aed9-e019ce24bdea None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:00.033051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ff2c6-b82f-4a8b-8d0a-ec5864e45bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:00.034220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ff2c6-b82f-4a8b-8d0a-ec5864e45bfc None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:00.034669 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ff2c6-b82f-4a8b-8d0a-ec5864e45bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:00.035175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ff2c6-b82f-4a8b-8d0a-ec5864e45bfc None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:00.043515 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ff2c6-b82f-4a8b-8d0a-ec5864e45bfc None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:00.046316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ff2c6-b82f-4a8b-8d0a-ec5864e45bfc None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:00.049385 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 144/272] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:11:59 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:00.050927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ff2c6-b82f-4a8b-8d0a-ec5864e45bfc None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:00.198401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a42b67-c37c-4ec5-aed9-e019ce24bdea None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:00.216583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ff2c6-b82f-4a8b-8d0a-ec5864e45bfc None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:00.238708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:00.269301 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da358af-6efc-4eba-99d0-ff3df7ba4457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:00.270069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b4cb-9841-4753-9650-abd3687bc9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:00.270692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da358af-6efc-4eba-99d0-ff3df7ba4457 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:00.271213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da358af-6efc-4eba-99d0-ff3df7ba4457 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:00.271674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da358af-6efc-4eba-99d0-ff3df7ba4457 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:00.272391 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b4cb-9841-4753-9650-abd3687bc9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:00.273099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b4cb-9841-4753-9650-abd3687bc9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:00.273547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da358af-6efc-4eba-99d0-ff3df7ba4457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:00.274196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b4cb-9841-4753-9650-abd3687bc9a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:00.276200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da358af-6efc-4eba-99d0-ff3df7ba4457 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:00.278376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 145/273] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:00.283918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da358af-6efc-4eba-99d0-ff3df7ba4457 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:00.295691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2b4cb-9841-4753-9650-abd3687bc9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e2c2b4cb-9841-4753-9650-abd3687bc9a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:00.313460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:00.315393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:00.317133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:00.321038 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 129/274] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:11:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:00.322786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b4cb-9841-4753-9650-abd3687bc9a7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:00.328864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:00.470948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da358af-6efc-4eba-99d0-ff3df7ba4457 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:00.519366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b4cb-9841-4753-9650-abd3687bc9a7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:00.530086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03070a-1364-4340-93a5-742aab42f6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:00.530911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03070a-1364-4340-93a5-742aab42f6df None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:00.531206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03070a-1364-4340-93a5-742aab42f6df None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:00.531624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03070a-1364-4340-93a5-742aab42f6df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:00.532550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03070a-1364-4340-93a5-742aab42f6df None admin] RBAC: Adding query filter params (role.id=f4461b279e86422b83e29521e7ca29d0, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user.id=83b8dc3f532542ddbe8018d329327c24) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:00.532982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03070a-1364-4340-93a5-742aab42f6df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:00.535168 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03070a-1364-4340-93a5-742aab42f6df None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:00.542380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 146/275] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:12:00 2022] GET /identity/v3/role_assignments?user.id=83b8dc3f532542ddbe8018d329327c24&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=f4461b279e86422b83e29521e7ca29d0 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:00.543773 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03070a-1364-4340-93a5-742aab42f6df None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:00.589222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f294d2-5ab9-469d-8b05-cd6feccfc3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:00.590351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f294d2-5ab9-469d-8b05-cd6feccfc3ca None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:00.590861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f294d2-5ab9-469d-8b05-cd6feccfc3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:00.591355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f294d2-5ab9-469d-8b05-cd6feccfc3ca None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:00.592626 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f294d2-5ab9-469d-8b05-cd6feccfc3ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:00.595208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f294d2-5ab9-469d-8b05-cd6feccfc3ca None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:00.623635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:00.633133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 130/276] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:12:00 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:12:00.740140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03070a-1364-4340-93a5-742aab42f6df None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:00.797792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f1c71-8d3d-46c2-9f1f-c3af3bbab03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:00.798403 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f1c71-8d3d-46c2-9f1f-c3af3bbab03b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:00.798720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f1c71-8d3d-46c2-9f1f-c3af3bbab03b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:00.798996 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f1c71-8d3d-46c2-9f1f-c3af3bbab03b None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:00.799775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f1c71-8d3d-46c2-9f1f-c3af3bbab03b None admin] RBAC: Adding query filter params (name=member) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:00.800080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f1c71-8d3d-46c2-9f1f-c3af3bbab03b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:00.801513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f1c71-8d3d-46c2-9f1f-c3af3bbab03b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:00.814940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 147/277] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:00.824168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f294d2-5ab9-469d-8b05-cd6feccfc3ca None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:00.924539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:01.021897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f294d2-5ab9-469d-8b05-cd6feccfc3ca None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:01.087275 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08d584-83cc-4e9b-9f70-42282f4a9b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:01.088071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08d584-83cc-4e9b-9f70-42282f4a9b28 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:01.088344 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08d584-83cc-4e9b-9f70-42282f4a9b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:01.088625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08d584-83cc-4e9b-9f70-42282f4a9b28 None admin] PATH_INFO: `/v3/users/2b0f328adf87416b83e86791a27ae441` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:01.115437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08d584-83cc-4e9b-9f70-42282f4a9b28 None admin] RBAC: Authorizing `identity:get_user(user_id=2b0f328adf87416b83e86791a27ae441)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:01.118220 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08d584-83cc-4e9b-9f70-42282f4a9b28 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:01.121827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 131/278] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:12:00 2022] GET /identity/v3/users/2b0f328adf87416b83e86791a27ae441 => generated 293 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:01.131204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f1c71-8d3d-46c2-9f1f-c3af3bbab03b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:01.271500 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f1c71-8d3d-46c2-9f1f-c3af3bbab03b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:01.288829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85d719-4721-4e6e-95fd-f78185a23170 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:01.289191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85d719-4721-4e6e-95fd-f78185a23170 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:01.289633 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85d719-4721-4e6e-95fd-f78185a23170 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:01.292175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 132/279] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:12:01.292592 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:01.299800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8328-aef3-4756-b38e-eaf9ea56a4af None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:01.300104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8328-aef3-4756-b38e-eaf9ea56a4af None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:01.300365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca8328-aef3-4756-b38e-eaf9ea56a4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:01.327993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732c5d3-5e6d-4773-8c2f-642825188670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:01.328661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732c5d3-5e6d-4773-8c2f-642825188670 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:01.328945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732c5d3-5e6d-4773-8c2f-642825188670 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:01.329219 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732c5d3-5e6d-4773-8c2f-642825188670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:01.330059 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732c5d3-5e6d-4773-8c2f-642825188670 None admin] RBAC: Adding query filter params (role.id=1249af5cf5054410a11b2d88c03fcd52, user.id=2b0f328adf87416b83e86791a27ae441, scope.system=all) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:01.330380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732c5d3-5e6d-4773-8c2f-642825188670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:01.331907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9732c5d3-5e6d-4773-8c2f-642825188670 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:01.333073 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c0ca8328-aef3-4756-b38e-eaf9ea56a4af None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:01.334479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c0ca8328-aef3-4756-b38e-eaf9ea56a4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:01.337968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 148/280] 10.222.0.93 () {56 vars in 1308 bytes} [Fri Jun 3 15:12:01 2022] GET /identity/v3/role_assignments?user.id=2b0f328adf87416b83e86791a27ae441&scope.system=all&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:01.375397 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901e7ff-c7a7-4bf8-b8ab-1e36a1271497 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:01.375804 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901e7ff-c7a7-4bf8-b8ab-1e36a1271497 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:01.375996 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901e7ff-c7a7-4bf8-b8ab-1e36a1271497 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:01.377690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 149/281] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:12:01.384011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114c478-c1fa-4f2e-84a5-5324efc8ba04 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:01.384394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114c478-c1fa-4f2e-84a5-5324efc8ba04 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:01.384660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114c478-c1fa-4f2e-84a5-5324efc8ba04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:01.403207 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c114c478-c1fa-4f2e-84a5-5324efc8ba04 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:01.404529 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c114c478-c1fa-4f2e-84a5-5324efc8ba04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:01.472202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 133/282] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:12:01.521865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 150/283] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:12:01.522284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca8328-aef3-4756-b38e-eaf9ea56a4af None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:01.546435 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114c478-c1fa-4f2e-84a5-5324efc8ba04 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:01.582145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:01.680867 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114c478-c1fa-4f2e-84a5-5324efc8ba04 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:01.690444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:01.693633 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca8328-aef3-4756-b38e-eaf9ea56a4af None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:01.723498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1759cb1-9054-494a-93be-186f30471013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:01.724586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1759cb1-9054-494a-93be-186f30471013 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:01.725025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1759cb1-9054-494a-93be-186f30471013 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:01.725454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1759cb1-9054-494a-93be-186f30471013 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:01.739170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1759cb1-9054-494a-93be-186f30471013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-f1759cb1-9054-494a-93be-186f30471013 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:01.751840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:01.753066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:12:01.754090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:01.757215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 151/284] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:01.763228 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1759cb1-9054-494a-93be-186f30471013 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:01.788553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642627df-e60c-4876-8659-cb4e0b24cf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:01.789531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642627df-e60c-4876-8659-cb4e0b24cf9b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:01.789849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642627df-e60c-4876-8659-cb4e0b24cf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:01.790387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642627df-e60c-4876-8659-cb4e0b24cf9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:01.803280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642627df-e60c-4876-8659-cb4e0b24cf9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-642627df-e60c-4876-8659-cb4e0b24cf9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:01.814735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:01.816517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:01.817977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 134/285] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:01.822416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642627df-e60c-4876-8659-cb4e0b24cf9b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:01.965887 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1759cb1-9054-494a-93be-186f30471013 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:02.027326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7a93e-6365-4dc9-9d2d-560dfa4312a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:02.034570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a93e-6365-4dc9-9d2d-560dfa4312a5 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:02.034570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a93e-6365-4dc9-9d2d-560dfa4312a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:02.034570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7a93e-6365-4dc9-9d2d-560dfa4312a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:02.034570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7a93e-6365-4dc9-9d2d-560dfa4312a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:02.034570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7a93e-6365-4dc9-9d2d-560dfa4312a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:02.035006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7a93e-6365-4dc9-9d2d-560dfa4312a5 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:02.052549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 152/286] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:02.061504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7a93e-6365-4dc9-9d2d-560dfa4312a5 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:02.116934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642627df-e60c-4876-8659-cb4e0b24cf9b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:02.138305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:02.146915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:02.262819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8fbf5-f869-4337-b3ce-e5b09cc54c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:02.265058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8fbf5-f869-4337-b3ce-e5b09cc54c57 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:02.265465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8fbf5-f869-4337-b3ce-e5b09cc54c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:02.265922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8fbf5-f869-4337-b3ce-e5b09cc54c57 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:02.267364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8fbf5-f869-4337-b3ce-e5b09cc54c57 None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:02.267860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8fbf5-f869-4337-b3ce-e5b09cc54c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:02.270294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8fbf5-f869-4337-b3ce-e5b09cc54c57 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:02.288953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 135/287] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 468 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:02.292062 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7a93e-6365-4dc9-9d2d-560dfa4312a5 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:02.296711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8fbf5-f869-4337-b3ce-e5b09cc54c57 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:02.391757 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9861d-39bf-4996-b352-f59c2cee2374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:02.392879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9861d-39bf-4996-b352-f59c2cee2374 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:02.393159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9861d-39bf-4996-b352-f59c2cee2374 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:02.393432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9861d-39bf-4996-b352-f59c2cee2374 None admin] PATH_INFO: `/v3/users/f2721a47fd464e4fa5ff08a701c1dede` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:02.395103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9861d-39bf-4996-b352-f59c2cee2374 None admin] RBAC: Authorizing `identity:get_user(user_id=f2721a47fd464e4fa5ff08a701c1dede)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:02.396563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9861d-39bf-4996-b352-f59c2cee2374 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:02.398580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 153/288] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:12:02 2022] GET /identity/v3/users/f2721a47fd464e4fa5ff08a701c1dede => generated 297 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:02.404892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9861d-39bf-4996-b352-f59c2cee2374 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:02.581134 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:02.591744 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9861d-39bf-4996-b352-f59c2cee2374 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:02.638276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8fbf5-f869-4337-b3ce-e5b09cc54c57 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:02.694325 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d750ee2-1a21-469c-ac82-5ad6d2fee1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:02.695277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d750ee2-1a21-469c-ac82-5ad6d2fee1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:02.695608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d750ee2-1a21-469c-ac82-5ad6d2fee1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:02.695887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d750ee2-1a21-469c-ac82-5ad6d2fee1d2 None admin] PATH_INFO: `/v3/users/97bd60be6f5a4095b17d73438c427592` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:02.697355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d750ee2-1a21-469c-ac82-5ad6d2fee1d2 None admin] RBAC: Authorizing `identity:get_user(user_id=97bd60be6f5a4095b17d73438c427592)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:02.698852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d750ee2-1a21-469c-ac82-5ad6d2fee1d2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:02.700786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 136/289] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:12:02 2022] GET /identity/v3/users/97bd60be6f5a4095b17d73438c427592 => generated 297 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:02.707227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d750ee2-1a21-469c-ac82-5ad6d2fee1d2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:02.739763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc0662-811c-42d6-aef7-66d0fee0730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:02.741218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc0662-811c-42d6-aef7-66d0fee0730a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:02.741709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc0662-811c-42d6-aef7-66d0fee0730a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:02.742227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc0662-811c-42d6-aef7-66d0fee0730a None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:02.747362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc0662-811c-42d6-aef7-66d0fee0730a None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:02.749428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc0662-811c-42d6-aef7-66d0fee0730a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:02.751281 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 154/290] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:02 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:02.760175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:02.767659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc0662-811c-42d6-aef7-66d0fee0730a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:02.891383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d750ee2-1a21-469c-ac82-5ad6d2fee1d2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:02.958583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d6edb-4e4a-48d5-b893-c59c39cbc667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:02.960351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d6edb-4e4a-48d5-b893-c59c39cbc667 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:02.960824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d6edb-4e4a-48d5-b893-c59c39cbc667 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:02.961495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d6edb-4e4a-48d5-b893-c59c39cbc667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:03.085595 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d6edb-4e4a-48d5-b893-c59c39cbc667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:03.087124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d6edb-4e4a-48d5-b893-c59c39cbc667 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:03.121094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc0662-811c-42d6-aef7-66d0fee0730a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:03.137813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:03.254900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 137/291] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:03.259541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6453d88-9ec0-4c5d-aab1-089e990fe108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:03.260603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6453d88-9ec0-4c5d-aab1-089e990fe108 None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:03.261010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6453d88-9ec0-4c5d-aab1-089e990fe108 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:03.261392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6453d88-9ec0-4c5d-aab1-089e990fe108 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/users/f2721a47fd464e4fa5ff08a701c1dede/roles/989320df8c4149cf918861e3ee5954d2` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:03.264061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6453d88-9ec0-4c5d-aab1-089e990fe108 None admin] RBAC: Authorizing `identity:create_grant(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user_id=f2721a47fd464e4fa5ff08a701c1dede, role_id=989320df8c4149cf918861e3ee5954d2)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:03.264854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d6edb-4e4a-48d5-b893-c59c39cbc667 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:03.265777 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6453d88-9ec0-4c5d-aab1-089e990fe108 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:03.296735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 155/292] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:12:02 2022] PUT /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/users/f2721a47fd464e4fa5ff08a701c1dede/roles/989320df8c4149cf918861e3ee5954d2 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:03.452319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d6edb-4e4a-48d5-b893-c59c39cbc667 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:03.504416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0be1e7-5307-47d4-bc1c-ab19f8129957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:03.505642 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0be1e7-5307-47d4-bc1c-ab19f8129957 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:03.506100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0be1e7-5307-47d4-bc1c-ab19f8129957 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:03.506547 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0be1e7-5307-47d4-bc1c-ab19f8129957 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:03.508054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0be1e7-5307-47d4-bc1c-ab19f8129957 None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:03.509760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0be1e7-5307-47d4-bc1c-ab19f8129957 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:03.512278 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 138/293] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:03 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:03.520040 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6453d88-9ec0-4c5d-aab1-089e990fe108 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:03.582555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:03.725283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6453d88-9ec0-4c5d-aab1-089e990fe108 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:03.780948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd703ea1-9ab0-45cc-a213-9958a0c92a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:03.781855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd703ea1-9ab0-45cc-a213-9958a0c92a04 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:03.782262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd703ea1-9ab0-45cc-a213-9958a0c92a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:03.782654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd703ea1-9ab0-45cc-a213-9958a0c92a04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:03.784467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd703ea1-9ab0-45cc-a213-9958a0c92a04 None admin] RBAC: Adding query filter params (role.id=1249af5cf5054410a11b2d88c03fcd52, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user.id=97bd60be6f5a4095b17d73438c427592) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:03.784967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd703ea1-9ab0-45cc-a213-9958a0c92a04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:03.787133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd703ea1-9ab0-45cc-a213-9958a0c92a04 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:03.798476 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 156/294] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:12:03 2022] GET /identity/v3/role_assignments?user.id=97bd60be6f5a4095b17d73438c427592&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:04.241035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994a848-5e54-4715-ab99-155bd39c5642 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:04.241804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994a848-5e54-4715-ab99-155bd39c5642 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:04.241804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994a848-5e54-4715-ab99-155bd39c5642 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:04.243392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 139/295] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:04.252127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a411f21-33d9-4af5-b974-8bd8d3035a50 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:04.252941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a411f21-33d9-4af5-b974-8bd8d3035a50 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:04.253483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a411f21-33d9-4af5-b974-8bd8d3035a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:04.286297 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-1a411f21-33d9-4af5-b974-8bd8d3035a50 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:04.288704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1a411f21-33d9-4af5-b974-8bd8d3035a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:04.425931 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 157/296] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:04.452051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994a848-5e54-4715-ab99-155bd39c5642 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:04.466290 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:04.680994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994a848-5e54-4715-ab99-155bd39c5642 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:04.737858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e7cd6-0d33-4190-b115-84d093e7eb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:04.738687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e7cd6-0d33-4190-b115-84d093e7eb54 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:04.738976 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e7cd6-0d33-4190-b115-84d093e7eb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:04.739320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e7cd6-0d33-4190-b115-84d093e7eb54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:04.753581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e7cd6-0d33-4190-b115-84d093e7eb54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-fb8e7cd6-0d33-4190-b115-84d093e7eb54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:04.765968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:04.768143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:04.769704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 140/297] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:04.780362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a411f21-33d9-4af5-b974-8bd8d3035a50 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:04.808927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:04.842871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653582b2-c08e-44b1-9f06-6294b7d05d4e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:04.843511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653582b2-c08e-44b1-9f06-6294b7d05d4e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:04.844023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653582b2-c08e-44b1-9f06-6294b7d05d4e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:04.845961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 141/298] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:04.851873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ca4c7-e8e5-4e2a-b953-ae7f43cfa5c0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:04.852472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ca4c7-e8e5-4e2a-b953-ae7f43cfa5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:04.852952 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ca4c7-e8e5-4e2a-b953-ae7f43cfa5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:04.873932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7d6ca4c7-e8e5-4e2a-b953-ae7f43cfa5c0 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:04.875586 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7d6ca4c7-e8e5-4e2a-b953-ae7f43cfa5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:04.961349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:04.966114 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a411f21-33d9-4af5-b974-8bd8d3035a50 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:04.978585 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 142/299] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:12:04.978585 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916a005-a637-4eed-9b74-7c9e5a453ea1 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:04.978585 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916a005-a637-4eed-9b74-7c9e5a453ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:04.978585 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916a005-a637-4eed-9b74-7c9e5a453ea1 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:04.981784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 143/300] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:04.988329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4636a0-0acb-421d-903c-6bf97b03f976 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:04.988760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4636a0-0acb-421d-903c-6bf97b03f976 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:04.989201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4636a0-0acb-421d-903c-6bf97b03f976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:05.011159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9d4636a0-0acb-421d-903c-6bf97b03f976 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:05.012977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9d4636a0-0acb-421d-903c-6bf97b03f976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:05.032568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c49b-851a-447c-9a5f-1a3bf4c2edd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:05.033857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c49b-851a-447c-9a5f-1a3bf4c2edd4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:05.034522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c49b-851a-447c-9a5f-1a3bf4c2edd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:05.035546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c49b-851a-447c-9a5f-1a3bf4c2edd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:05.037169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c49b-851a-447c-9a5f-1a3bf4c2edd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:05.037479 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c49b-851a-447c-9a5f-1a3bf4c2edd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:05.039307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c49b-851a-447c-9a5f-1a3bf4c2edd4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:05.051904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 158/301] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:05.052492 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c49b-851a-447c-9a5f-1a3bf4c2edd4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:05.146583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 144/302] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:05.146583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4636a0-0acb-421d-903c-6bf97b03f976 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:05.198856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:05.241919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c49b-851a-447c-9a5f-1a3bf4c2edd4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:05.309967 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4636a0-0acb-421d-903c-6bf97b03f976 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:05.312337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5551b-a5f4-4459-9649-9b1bd8b5c319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:05.313502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5551b-a5f4-4459-9649-9b1bd8b5c319 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:05.313883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5551b-a5f4-4459-9649-9b1bd8b5c319 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:05.315346 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5551b-a5f4-4459-9649-9b1bd8b5c319 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:05.332993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5551b-a5f4-4459-9649-9b1bd8b5c319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-73b5551b-a5f4-4459-9649-9b1bd8b5c319 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:05.341954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:05.347271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 159/303] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:05.349056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5551b-a5f4-4459-9649-9b1bd8b5c319 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:05.356450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:05.371297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be024d69-a10b-4d5a-9304-e3c2aaf14960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:05.372191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be024d69-a10b-4d5a-9304-e3c2aaf14960 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:05.372473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be024d69-a10b-4d5a-9304-e3c2aaf14960 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:05.372751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be024d69-a10b-4d5a-9304-e3c2aaf14960 None admin] PATH_INFO: `/v3/users/2b0f328adf87416b83e86791a27ae441` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:05.374786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be024d69-a10b-4d5a-9304-e3c2aaf14960 None admin] RBAC: Authorizing `identity:get_user(user_id=2b0f328adf87416b83e86791a27ae441)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:05.376293 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be024d69-a10b-4d5a-9304-e3c2aaf14960 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:05.378495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 145/304] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:12:05 2022] GET /identity/v3/users/2b0f328adf87416b83e86791a27ae441 => generated 293 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:05.383898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be024d69-a10b-4d5a-9304-e3c2aaf14960 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:05.518745 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5551b-a5f4-4459-9649-9b1bd8b5c319 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:05.545844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be024d69-a10b-4d5a-9304-e3c2aaf14960 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:05.577399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d967a5-b97d-4179-afe9-3ea393c8879f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:05.578224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d967a5-b97d-4179-afe9-3ea393c8879f None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:05.578763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d967a5-b97d-4179-afe9-3ea393c8879f None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:05.579264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d967a5-b97d-4179-afe9-3ea393c8879f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:05.580762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d967a5-b97d-4179-afe9-3ea393c8879f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:05.582422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d967a5-b97d-4179-afe9-3ea393c8879f None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:05.589128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 160/305] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:05.589128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d967a5-b97d-4179-afe9-3ea393c8879f None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:05.602075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:05.603516 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd86c1-adfb-49b2-9fd9-8ff27380aa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:05.604217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd86c1-adfb-49b2-9fd9-8ff27380aa0c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:05.604970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd86c1-adfb-49b2-9fd9-8ff27380aa0c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:05.604970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bd86c1-adfb-49b2-9fd9-8ff27380aa0c None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:05.605633 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd86c1-adfb-49b2-9fd9-8ff27380aa0c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:05.605974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd86c1-adfb-49b2-9fd9-8ff27380aa0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:05.607837 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bd86c1-adfb-49b2-9fd9-8ff27380aa0c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:05.627874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 146/306] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:05.627874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd86c1-adfb-49b2-9fd9-8ff27380aa0c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:05.736814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:05.779400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d967a5-b97d-4179-afe9-3ea393c8879f None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:05.811317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bd86c1-adfb-49b2-9fd9-8ff27380aa0c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:05.836520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48901534-ba4c-4e9e-9dec-3469179b5483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:05.837603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48901534-ba4c-4e9e-9dec-3469179b5483 None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:05.837896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48901534-ba4c-4e9e-9dec-3469179b5483 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:05.838329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48901534-ba4c-4e9e-9dec-3469179b5483 None admin] PATH_INFO: `/v3/system/users/2b0f328adf87416b83e86791a27ae441/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:05.841069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48901534-ba4c-4e9e-9dec-3469179b5483 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2b0f328adf87416b83e86791a27ae441, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:05.842657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48901534-ba4c-4e9e-9dec-3469179b5483 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:05.858913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 161/307] 10.222.0.93 () {58 vars in 1334 bytes} [Fri Jun 3 15:12:05 2022] PUT /identity/v3/system/users/2b0f328adf87416b83e86791a27ae441/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:05.871474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48901534-ba4c-4e9e-9dec-3469179b5483 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:05.871783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186474f-9805-4531-a73f-8a296c3106b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:05.872598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186474f-9805-4531-a73f-8a296c3106b4 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:05.873495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186474f-9805-4531-a73f-8a296c3106b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:05.873877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186474f-9805-4531-a73f-8a296c3106b4 None admin] PATH_INFO: `/v3/groups` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:05.882829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186474f-9805-4531-a73f-8a296c3106b4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:05.882829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186474f-9805-4531-a73f-8a296c3106b4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:05.911519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 147/308] 10.222.0.93 () {60 vars in 1134 bytes} [Fri Jun 3 15:12:05 2022] POST /identity/v3/groups => generated 229 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:12:05.988753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:06.035203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48901534-ba4c-4e9e-9dec-3469179b5483 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:06.110842 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63cfff-27aa-4e23-8da0-82e5c294cf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:06.112460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63cfff-27aa-4e23-8da0-82e5c294cf08 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:06.113095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63cfff-27aa-4e23-8da0-82e5c294cf08 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:06.113407 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63cfff-27aa-4e23-8da0-82e5c294cf08 None admin] PATH_INFO: `/v3/users/095170e111c24ffebab6232fedd8b26b` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:06.144417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63cfff-27aa-4e23-8da0-82e5c294cf08 None admin] RBAC: Authorizing `identity:get_user(user_id=095170e111c24ffebab6232fedd8b26b)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:06.147366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63cfff-27aa-4e23-8da0-82e5c294cf08 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:06.149520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 162/309] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:12:05 2022] GET /identity/v3/users/095170e111c24ffebab6232fedd8b26b => generated 293 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:06.156647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186474f-9805-4531-a73f-8a296c3106b4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:06.339205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186474f-9805-4531-a73f-8a296c3106b4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:06.379081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:06.425030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db517127-c9be-40db-bc01-1fd586365531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:06.426768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db517127-c9be-40db-bc01-1fd586365531 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:06.427601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db517127-c9be-40db-bc01-1fd586365531 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:06.428386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db517127-c9be-40db-bc01-1fd586365531 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:06.430460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db517127-c9be-40db-bc01-1fd586365531 None admin] RBAC: Adding query filter params (role.id=989320df8c4149cf918861e3ee5954d2, user.id=095170e111c24ffebab6232fedd8b26b, scope.system=all) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:06.431587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db517127-c9be-40db-bc01-1fd586365531 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:06.434849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db517127-c9be-40db-bc01-1fd586365531 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:06.445813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 148/310] 10.222.0.93 () {56 vars in 1308 bytes} [Fri Jun 3 15:12:06 2022] GET /identity/v3/role_assignments?user.id=095170e111c24ffebab6232fedd8b26b&scope.system=all&role.id=989320df8c4149cf918861e3ee5954d2 => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:06.493076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ced75d-5b7d-429b-9ff5-83b02c941cfc None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:06.493888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ced75d-5b7d-429b-9ff5-83b02c941cfc None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:06.493888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ced75d-5b7d-429b-9ff5-83b02c941cfc None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:06.495383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 163/311] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:06.502443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f89f24-de70-4551-81e4-2a331e2c34f4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:06.502892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f89f24-de70-4551-81e4-2a331e2c34f4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:06.503349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f89f24-de70-4551-81e4-2a331e2c34f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:06.528877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c8f89f24-de70-4551-81e4-2a331e2c34f4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:06.531528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c8f89f24-de70-4551-81e4-2a331e2c34f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:06.643013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 149/312] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:06.668345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ced75d-5b7d-429b-9ff5-83b02c941cfc None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:06.799454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ced75d-5b7d-429b-9ff5-83b02c941cfc None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:06.843487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:06.851032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38705e-1245-460d-881b-591e2a89cd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:06.852276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38705e-1245-460d-881b-591e2a89cd0e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:06.852558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38705e-1245-460d-881b-591e2a89cd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:06.853016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38705e-1245-460d-881b-591e2a89cd0e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:06.866675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38705e-1245-460d-881b-591e2a89cd0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ee38705e-1245-460d-881b-591e2a89cd0e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:06.879140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:06.880835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:06.882260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 164/313] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:06.892533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f89f24-de70-4551-81e4-2a331e2c34f4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:07.023678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f89f24-de70-4551-81e4-2a331e2c34f4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:07.035779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:07.065766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27bf48-28f5-4901-b963-761ad77ac85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:07.066465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27bf48-28f5-4901-b963-761ad77ac85c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:07.066778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27bf48-28f5-4901-b963-761ad77ac85c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:07.067053 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27bf48-28f5-4901-b963-761ad77ac85c None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:07.067817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27bf48-28f5-4901-b963-761ad77ac85c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:07.068121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27bf48-28f5-4901-b963-761ad77ac85c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:07.069637 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27bf48-28f5-4901-b963-761ad77ac85c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:07.081046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 150/314] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:07.090221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38705e-1245-460d-881b-591e2a89cd0e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:07.251252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38705e-1245-460d-881b-591e2a89cd0e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:07.302559 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086eacaa-4470-4681-9672-85427abfc1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:07.303927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eacaa-4470-4681-9672-85427abfc1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:07.304409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eacaa-4470-4681-9672-85427abfc1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:07.304891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086eacaa-4470-4681-9672-85427abfc1c3 None admin] PATH_INFO: `/v3/users/f2721a47fd464e4fa5ff08a701c1dede` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:07.306767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086eacaa-4470-4681-9672-85427abfc1c3 None admin] RBAC: Authorizing `identity:get_user(user_id=f2721a47fd464e4fa5ff08a701c1dede)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:07.308622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086eacaa-4470-4681-9672-85427abfc1c3 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:07.310925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 165/315] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:12:07 2022] GET /identity/v3/users/f2721a47fd464e4fa5ff08a701c1dede => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:07.316547 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27bf48-28f5-4901-b963-761ad77ac85c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:07.429996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27bf48-28f5-4901-b963-761ad77ac85c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:07.473828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf037d-4bcc-4019-9162-b8a5171cbfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:07.474490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf037d-4bcc-4019-9162-b8a5171cbfb4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:07.474806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf037d-4bcc-4019-9162-b8a5171cbfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:07.475083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf037d-4bcc-4019-9162-b8a5171cbfb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:07.518063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:07.553359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf037d-4bcc-4019-9162-b8a5171cbfb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:07.554801 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbf037d-4bcc-4019-9162-b8a5171cbfb4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:07.661312 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 151/316] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:07.667198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086eacaa-4470-4681-9672-85427abfc1c3 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:07.733783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:07.784973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086eacaa-4470-4681-9672-85427abfc1c3 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:07.824871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa42f70-ef94-4c3d-bfdc-56dc8bbc8571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:07.825859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa42f70-ef94-4c3d-bfdc-56dc8bbc8571 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:07.826699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa42f70-ef94-4c3d-bfdc-56dc8bbc8571 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:07.827058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa42f70-ef94-4c3d-bfdc-56dc8bbc8571 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:07.828730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa42f70-ef94-4c3d-bfdc-56dc8bbc8571 None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:07.830177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa42f70-ef94-4c3d-bfdc-56dc8bbc8571 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:07.831908 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 166/317] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:07 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:07.837873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf037d-4bcc-4019-9162-b8a5171cbfb4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:07.911672 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:07.949181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf037d-4bcc-4019-9162-b8a5171cbfb4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:08.002219 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809d0d5-2efd-431e-ba9d-63be70c14920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:08.003151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809d0d5-2efd-431e-ba9d-63be70c14920 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:08.003560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809d0d5-2efd-431e-ba9d-63be70c14920 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:08.003848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809d0d5-2efd-431e-ba9d-63be70c14920 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:08.004814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809d0d5-2efd-431e-ba9d-63be70c14920 None admin] RBAC: Adding query filter params (role.id=989320df8c4149cf918861e3ee5954d2, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9, user.id=f2721a47fd464e4fa5ff08a701c1dede) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:08.005132 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809d0d5-2efd-431e-ba9d-63be70c14920 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:08.006805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809d0d5-2efd-431e-ba9d-63be70c14920 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:08.024927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 152/318] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:12:07 2022] GET /identity/v3/role_assignments?user.id=f2721a47fd464e4fa5ff08a701c1dede&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=989320df8c4149cf918861e3ee5954d2 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:08.822934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd526ef-31b6-4897-aa44-55619e417052 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:08.823554 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd526ef-31b6-4897-aa44-55619e417052 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:08.823554 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd526ef-31b6-4897-aa44-55619e417052 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:08.825325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 167/319] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:08.832943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aacb31-ba19-43af-b00f-7c580e0264a9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:08.833457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aacb31-ba19-43af-b00f-7c580e0264a9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:08.833533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aacb31-ba19-43af-b00f-7c580e0264a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:08.857447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-85aacb31-ba19-43af-b00f-7c580e0264a9 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:08.858906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-85aacb31-ba19-43af-b00f-7c580e0264a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:08.969900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 153/320] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:09.001855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd526ef-31b6-4897-aa44-55619e417052 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:09.020916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d106-bc4c-4501-a5b6-660513febfd9 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.021660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d106-bc4c-4501-a5b6-660513febfd9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.022150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5d106-bc4c-4501-a5b6-660513febfd9 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.024816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 154/321] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:09.032743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbead2-9f83-4e15-ad21-af4a07b05cfa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.033139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbead2-9f83-4e15-ad21-af4a07b05cfa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.033312 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbead2-9f83-4e15-ad21-af4a07b05cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.056134 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-44dbead2-9f83-4e15-ad21-af4a07b05cfa None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:09.057336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-44dbead2-9f83-4e15-ad21-af4a07b05cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:09.140810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:09.148800 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd526ef-31b6-4897-aa44-55619e417052 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:09.151514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 155/322] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:09.176306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbead2-9f83-4e15-ad21-af4a07b05cfa None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:09.196036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:09.206684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b026fd-f291-4b10-a086-1a9399e3279e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:09.207490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b026fd-f291-4b10-a086-1a9399e3279e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.207774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b026fd-f291-4b10-a086-1a9399e3279e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.208154 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b026fd-f291-4b10-a086-1a9399e3279e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.220422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b026fd-f291-4b10-a086-1a9399e3279e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-f8b026fd-f291-4b10-a086-1a9399e3279e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:09.228257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:09.229935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 168/323] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:09.231578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377c359-1bf8-4181-a0b8-fb1aa4c8b9f7 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.233621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377c359-1bf8-4181-a0b8-fb1aa4c8b9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.233621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377c359-1bf8-4181-a0b8-fb1aa4c8b9f7 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.233621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 169/324] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:09.241202 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94053bc9-1434-426f-9e09-279a9660c53d None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.241490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94053bc9-1434-426f-9e09-279a9660c53d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.241777 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94053bc9-1434-426f-9e09-279a9660c53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.259415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-94053bc9-1434-426f-9e09-279a9660c53d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:09.260638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-94053bc9-1434-426f-9e09-279a9660c53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:09.311490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbead2-9f83-4e15-ad21-af4a07b05cfa None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:09.339847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 170/325] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:09.340722 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94053bc9-1434-426f-9e09-279a9660c53d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:09.355279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90d282-8d3f-4804-885a-f2d22424d3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:09.356344 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90d282-8d3f-4804-885a-f2d22424d3be None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.356626 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90d282-8d3f-4804-885a-f2d22424d3be None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.356904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90d282-8d3f-4804-885a-f2d22424d3be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.358803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90d282-8d3f-4804-885a-f2d22424d3be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:09.360388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90d282-8d3f-4804-885a-f2d22424d3be None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:09.374950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 156/326] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:09.374950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90d282-8d3f-4804-885a-f2d22424d3be None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:09.452730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:09.468803 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94053bc9-1434-426f-9e09-279a9660c53d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:09.489818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:09.499235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90d282-8d3f-4804-885a-f2d22424d3be None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:09.521989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f6f42-555b-4cbd-93a5-34f82c6f43cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:09.522756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f6f42-555b-4cbd-93a5-34f82c6f43cb None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.523036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f6f42-555b-4cbd-93a5-34f82c6f43cb None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.523387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f6f42-555b-4cbd-93a5-34f82c6f43cb None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.524133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f6f42-555b-4cbd-93a5-34f82c6f43cb None admin] RBAC: Adding query filter params (name=member) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:09.524431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f6f42-555b-4cbd-93a5-34f82c6f43cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:09.525839 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f6f42-555b-4cbd-93a5-34f82c6f43cb None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:09.537447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 171/327] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:09.538322 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f6f42-555b-4cbd-93a5-34f82c6f43cb None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:09.538474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eecb73-bfc8-4834-863d-faf91b0c4e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:09.539495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eecb73-bfc8-4834-863d-faf91b0c4e35 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.540020 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eecb73-bfc8-4834-863d-faf91b0c4e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.540348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eecb73-bfc8-4834-863d-faf91b0c4e35 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.553478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eecb73-bfc8-4834-863d-faf91b0c4e35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c1eecb73-bfc8-4834-863d-faf91b0c4e35 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:09.561366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:09.563258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 157/328] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:09.564252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eecb73-bfc8-4834-863d-faf91b0c4e35 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:09.661584 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f6f42-555b-4cbd-93a5-34f82c6f43cb None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:09.677042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eecb73-bfc8-4834-863d-faf91b0c4e35 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:09.705197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:09.706672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7aefaa-9e94-496b-bbe0-5f372ad78e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:09.707753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aefaa-9e94-496b-bbe0-5f372ad78e52 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.708071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aefaa-9e94-496b-bbe0-5f372ad78e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.708308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aefaa-9e94-496b-bbe0-5f372ad78e52 None admin] PATH_INFO: `/v3/groups` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.709006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7aefaa-9e94-496b-bbe0-5f372ad78e52 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:09.710585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7aefaa-9e94-496b-bbe0-5f372ad78e52 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:09.719711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e3ab1-7cbf-4ea4-84f7-4ee000733a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:09.720696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e3ab1-7cbf-4ea4-84f7-4ee000733a19 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.720973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e3ab1-7cbf-4ea4-84f7-4ee000733a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.721245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e3ab1-7cbf-4ea4-84f7-4ee000733a19 None admin] PATH_INFO: `/v3/users/2b0f328adf87416b83e86791a27ae441` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.722580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e3ab1-7cbf-4ea4-84f7-4ee000733a19 None admin] RBAC: Authorizing `identity:get_user(user_id=2b0f328adf87416b83e86791a27ae441)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:09.724121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e3ab1-7cbf-4ea4-84f7-4ee000733a19 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:09.726142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 158/329] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:12:09 2022] GET /identity/v3/users/2b0f328adf87416b83e86791a27ae441 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:09.726850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e3ab1-7cbf-4ea4-84f7-4ee000733a19 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:09.746657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 172/330] 10.222.0.93 () {60 vars in 1134 bytes} [Fri Jun 3 15:12:09 2022] POST /identity/v3/groups => generated 226 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:12:09.746657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7aefaa-9e94-496b-bbe0-5f372ad78e52 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:09.794616 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:09.893124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e3ab1-7cbf-4ea4-84f7-4ee000733a19 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:09.908743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7aefaa-9e94-496b-bbe0-5f372ad78e52 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:09.949085 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38ab8e-ad6b-4d6f-b841-dbf99a5f9f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:09.949780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38ab8e-ad6b-4d6f-b841-dbf99a5f9f5d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.950049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38ab8e-ad6b-4d6f-b841-dbf99a5f9f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.950333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b38ab8e-ad6b-4d6f-b841-dbf99a5f9f5d None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.951680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b38ab8e-ad6b-4d6f-b841-dbf99a5f9f5d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:09.951987 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b38ab8e-ad6b-4d6f-b841-dbf99a5f9f5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:09.953388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b38ab8e-ad6b-4d6f-b841-dbf99a5f9f5d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:09.963991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 159/331] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:09.965528 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e42623-80ef-428e-92dd-aab69baa7ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:09.967662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e42623-80ef-428e-92dd-aab69baa7ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:09.967662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e42623-80ef-428e-92dd-aab69baa7ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:09.967662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e42623-80ef-428e-92dd-aab69baa7ea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:09.968069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e42623-80ef-428e-92dd-aab69baa7ea7 None admin] RBAC: Adding query filter params (role.id=1249af5cf5054410a11b2d88c03fcd52, user.id=2b0f328adf87416b83e86791a27ae441, scope.system=all) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:09.968069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e42623-80ef-428e-92dd-aab69baa7ea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:09.970370 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e42623-80ef-428e-92dd-aab69baa7ea7 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:09.970524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38ab8e-ad6b-4d6f-b841-dbf99a5f9f5d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:09.987217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 173/332] 10.222.0.93 () {56 vars in 1308 bytes} [Fri Jun 3 15:12:09 2022] GET /identity/v3/role_assignments?user.id=2b0f328adf87416b83e86791a27ae441&scope.system=all&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 510 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:10.069841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:10.161458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b38ab8e-ad6b-4d6f-b841-dbf99a5f9f5d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:10.210236 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2290aa2-8206-4445-953e-b1c5a54d3000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:10.211698 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2290aa2-8206-4445-953e-b1c5a54d3000 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:10.212237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2290aa2-8206-4445-953e-b1c5a54d3000 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:10.212369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2290aa2-8206-4445-953e-b1c5a54d3000 None admin] PATH_INFO: `/v3/users/095170e111c24ffebab6232fedd8b26b` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:10.214500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2290aa2-8206-4445-953e-b1c5a54d3000 None admin] RBAC: Authorizing `identity:get_user(user_id=095170e111c24ffebab6232fedd8b26b)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:10.216183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2290aa2-8206-4445-953e-b1c5a54d3000 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:10.219213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 160/333] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:12:09 2022] GET /identity/v3/users/095170e111c24ffebab6232fedd8b26b => generated 293 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:10.226733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e42623-80ef-428e-92dd-aab69baa7ea7 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:10.413149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:10.425757 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e42623-80ef-428e-92dd-aab69baa7ea7 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:10.474017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed9e93-7ae3-490e-b952-019162512038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:10.474936 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed9e93-7ae3-490e-b952-019162512038 None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:10.475218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed9e93-7ae3-490e-b952-019162512038 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:10.475498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed9e93-7ae3-490e-b952-019162512038 None admin] PATH_INFO: `/v3/system/users/095170e111c24ffebab6232fedd8b26b/roles/989320df8c4149cf918861e3ee5954d2` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:10.477122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed9e93-7ae3-490e-b952-019162512038 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=095170e111c24ffebab6232fedd8b26b, role_id=989320df8c4149cf918861e3ee5954d2)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:10.478590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed9e93-7ae3-490e-b952-019162512038 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:10.488074 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 174/334] 10.222.0.93 () {58 vars in 1334 bytes} [Fri Jun 3 15:12:10 2022] PUT /identity/v3/system/users/095170e111c24ffebab6232fedd8b26b/roles/989320df8c4149cf918861e3ee5954d2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:13.686074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99854b4-f768-41b2-a11f-63b560679baf None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:13.687543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99854b4-f768-41b2-a11f-63b560679baf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:13.688264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99854b4-f768-41b2-a11f-63b560679baf None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:13.690879 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 161/335] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:13.700557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf781a13-61c2-4dcf-9109-96ee1bd2e2e2 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:13.702392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf781a13-61c2-4dcf-9109-96ee1bd2e2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:13.703135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf781a13-61c2-4dcf-9109-96ee1bd2e2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:13.739621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-bf781a13-61c2-4dcf-9109-96ee1bd2e2e2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:13.740847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-bf781a13-61c2-4dcf-9109-96ee1bd2e2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:13.831188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e89a0-e702-486f-907d-53605b403d18 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:13.832118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e89a0-e702-486f-907d-53605b403d18 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:13.832872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e89a0-e702-486f-907d-53605b403d18 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:13.835632 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 162/336] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:13.843709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036f206-6d1c-42c7-8df0-27df54bffb93 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:13.844268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036f206-6d1c-42c7-8df0-27df54bffb93 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:13.844753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036f206-6d1c-42c7-8df0-27df54bffb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:13.884326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0036f206-6d1c-42c7-8df0-27df54bffb93 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:13.886396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0036f206-6d1c-42c7-8df0-27df54bffb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:13.891439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 175/337] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:13.998930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf781a13-61c2-4dcf-9109-96ee1bd2e2e2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:14.037333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 163/338] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:12:14.093304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036f206-6d1c-42c7-8df0-27df54bffb93 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:14.224409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:14.232402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:14.273260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf781a13-61c2-4dcf-9109-96ee1bd2e2e2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:14.346937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036f206-6d1c-42c7-8df0-27df54bffb93 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:14.371398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52114bcb-b514-47f3-8786-cb0c1c0578d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:14.372228 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52114bcb-b514-47f3-8786-cb0c1c0578d4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:14.372546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52114bcb-b514-47f3-8786-cb0c1c0578d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:14.372807 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52114bcb-b514-47f3-8786-cb0c1c0578d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:14.404502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52114bcb-b514-47f3-8786-cb0c1c0578d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-52114bcb-b514-47f3-8786-cb0c1c0578d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:14.420387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:14.421589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:14.422772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 176/339] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:14.438481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52114bcb-b514-47f3-8786-cb0c1c0578d4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:14.444614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cde82-1314-43b4-92a6-33abf2d3c39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:14.445605 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cde82-1314-43b4-92a6-33abf2d3c39d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:14.446099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cde82-1314-43b4-92a6-33abf2d3c39d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:14.446502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cde82-1314-43b4-92a6-33abf2d3c39d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:14.475627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cde82-1314-43b4-92a6-33abf2d3c39d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-994cde82-1314-43b4-92a6-33abf2d3c39d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:14.484878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:14.486081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:14.487418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 164/340] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:12:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:14.494802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cde82-1314-43b4-92a6-33abf2d3c39d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:14.743305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52114bcb-b514-47f3-8786-cb0c1c0578d4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:14.774138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cde82-1314-43b4-92a6-33abf2d3c39d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:14.793292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:14.861936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec389b25-5e48-4eeb-ae7f-042ba2aa7542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:14.864049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec389b25-5e48-4eeb-ae7f-042ba2aa7542 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:14.865016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd0b85-6806-41be-ac96-5260b94f061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:14.865958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd0b85-6806-41be-ac96-5260b94f061b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:14.866550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd0b85-6806-41be-ac96-5260b94f061b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:14.867053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd0b85-6806-41be-ac96-5260b94f061b None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:14.868064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd0b85-6806-41be-ac96-5260b94f061b None admin] RBAC: Adding query filter params (name=member) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:14.868746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec389b25-5e48-4eeb-ae7f-042ba2aa7542 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:14.869302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd0b85-6806-41be-ac96-5260b94f061b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:14.869785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec389b25-5e48-4eeb-ae7f-042ba2aa7542 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:14.871656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cd0b85-6806-41be-ac96-5260b94f061b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:14.872871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec389b25-5e48-4eeb-ae7f-042ba2aa7542 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:14.873660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec389b25-5e48-4eeb-ae7f-042ba2aa7542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:14.876393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec389b25-5e48-4eeb-ae7f-042ba2aa7542 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:14.902743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 165/341] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:14.905496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e48d4-af2f-428a-b424-d3fce5a7c40d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:14.906140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e48d4-af2f-428a-b424-d3fce5a7c40d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:14.906665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e48d4-af2f-428a-b424-d3fce5a7c40d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:14.908715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 166/342] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:14.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 177/343] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:14.915905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d527250-eafe-4086-8a6a-9ce2327e18af None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:14.916494 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d527250-eafe-4086-8a6a-9ce2327e18af None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:14.916603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd0b85-6806-41be-ac96-5260b94f061b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:14.917315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d527250-eafe-4086-8a6a-9ce2327e18af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:14.942321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0d527250-eafe-4086-8a6a-9ce2327e18af None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:14.943876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0d527250-eafe-4086-8a6a-9ce2327e18af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:14.944992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:15.034886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 167/344] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:15.037957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b905026-73c6-4774-a403-26eb8630c989 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:15.038486 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b905026-73c6-4774-a403-26eb8630c989 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:15.039001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b905026-73c6-4774-a403-26eb8630c989 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:15.040822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 168/345] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:15.041739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b905026-73c6-4774-a403-26eb8630c989 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:15.087912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd0b85-6806-41be-ac96-5260b94f061b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:15.146507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58040bb1-3e70-40e1-b699-ae95d83e26eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:15.147343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58040bb1-3e70-40e1-b699-ae95d83e26eb None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:15.147589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58040bb1-3e70-40e1-b699-ae95d83e26eb None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:15.147873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58040bb1-3e70-40e1-b699-ae95d83e26eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:15.157912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b905026-73c6-4774-a403-26eb8630c989 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:15.162007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:15.204232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ffc20-13b6-4a10-a68c-8841aef2e81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:15.206060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ffc20-13b6-4a10-a68c-8841aef2e81a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:15.206609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ffc20-13b6-4a10-a68c-8841aef2e81a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:15.207083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ffc20-13b6-4a10-a68c-8841aef2e81a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:15.250890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58040bb1-3e70-40e1-b699-ae95d83e26eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:15.252188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58040bb1-3e70-40e1-b699-ae95d83e26eb None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:15.271458 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:15.303049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ffc20-13b6-4a10-a68c-8841aef2e81a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:15.305620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ffc20-13b6-4a10-a68c-8841aef2e81a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:15.397400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 178/346] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 03 15:12:15.399897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d427d2b-3c46-4074-818e-f30d928b783c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:15.400348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d427d2b-3c46-4074-818e-f30d928b783c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:15.400799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d427d2b-3c46-4074-818e-f30d928b783c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:15.429377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-3d427d2b-3c46-4074-818e-f30d928b783c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:15.430688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3d427d2b-3c46-4074-818e-f30d928b783c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:15.449885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 169/347] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:15.451483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ffc20-13b6-4a10-a68c-8841aef2e81a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:15.455743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:15.555271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 179/348] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:15.555271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47894b47-f6ff-4ce5-a50c-7de320c4a54c None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:15.555271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47894b47-f6ff-4ce5-a50c-7de320c4a54c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:15.555271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47894b47-f6ff-4ce5-a50c-7de320c4a54c None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:15.555271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 180/349] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:15.555953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47894b47-f6ff-4ce5-a50c-7de320c4a54c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:15.617039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:15.626305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ffc20-13b6-4a10-a68c-8841aef2e81a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:15.670116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236adde2-25f5-4c21-8a0a-fc192206f34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:15.672705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236adde2-25f5-4c21-8a0a-fc192206f34e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:15.672992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236adde2-25f5-4c21-8a0a-fc192206f34e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:15.673270 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236adde2-25f5-4c21-8a0a-fc192206f34e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:15.689123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236adde2-25f5-4c21-8a0a-fc192206f34e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-236adde2-25f5-4c21-8a0a-fc192206f34e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:15.695208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:15.698342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 170/350] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:15.699864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236adde2-25f5-4c21-8a0a-fc192206f34e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:15.724819 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47894b47-f6ff-4ce5-a50c-7de320c4a54c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:15.782584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:15.790805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ce95f-48ec-4ed4-9268-39e6542ea4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:15.793004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ce95f-48ec-4ed4-9268-39e6542ea4ab None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:15.793320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ce95f-48ec-4ed4-9268-39e6542ea4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:15.793637 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ce95f-48ec-4ed4-9268-39e6542ea4ab None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:15.795897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ce95f-48ec-4ed4-9268-39e6542ea4ab None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:15.798747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ce95f-48ec-4ed4-9268-39e6542ea4ab None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:15.802700 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 181/351] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:15 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:15.805365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89eb7-2d93-4b36-ba37-a00c01d6027d None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:15.806160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89eb7-2d93-4b36-ba37-a00c01d6027d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:15.806953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec89eb7-2d93-4b36-ba37-a00c01d6027d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:15.842042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-6ec89eb7-2d93-4b36-ba37-a00c01d6027d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:15.843514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6ec89eb7-2d93-4b36-ba37-a00c01d6027d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:15.861130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236adde2-25f5-4c21-8a0a-fc192206f34e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:15.899481 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8188c-bd09-4702-ba74-efe92ebfb8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:15.900374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8188c-bd09-4702-ba74-efe92ebfb8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:15.900648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8188c-bd09-4702-ba74-efe92ebfb8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:15.901130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8188c-bd09-4702-ba74-efe92ebfb8f5 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:15.902156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8188c-bd09-4702-ba74-efe92ebfb8f5 None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:15.904304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8188c-bd09-4702-ba74-efe92ebfb8f5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:15.906032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 171/352] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:15 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:15.907030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8188c-bd09-4702-ba74-efe92ebfb8f5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:15.954399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 182/353] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:15.955815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec89eb7-2d93-4b36-ba37-a00c01d6027d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:16.022918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:16.088785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8188c-bd09-4702-ba74-efe92ebfb8f5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:16.114381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec89eb7-2d93-4b36-ba37-a00c01d6027d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:16.131800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:16.149623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdd449-e2df-47c5-ba48-5d262f0a80b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:16.150440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdd449-e2df-47c5-ba48-5d262f0a80b9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:16.150828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdd449-e2df-47c5-ba48-5d262f0a80b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:16.151105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fdd449-e2df-47c5-ba48-5d262f0a80b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:16.171233 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cee2b-f795-40a2-88b4-ad046fef2b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:16.171633 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fdd449-e2df-47c5-ba48-5d262f0a80b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:16.172160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cee2b-f795-40a2-88b4-ad046fef2b0c None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:16.172629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cee2b-f795-40a2-88b4-ad046fef2b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:16.172958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cee2b-f795-40a2-88b4-ad046fef2b0c None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:16.173680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cee2b-f795-40a2-88b4-ad046fef2b0c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:16.174060 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cee2b-f795-40a2-88b4-ad046fef2b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:16.175402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cee2b-f795-40a2-88b4-ad046fef2b0c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-79fdd449-e2df-47c5-ba48-5d262f0a80b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:16.180835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:16.181981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:16.182823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:16.185981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 172/354] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:16.185981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdd449-e2df-47c5-ba48-5d262f0a80b9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:16.188937 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 183/355] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:16.190630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f791ff-bb84-4ef1-ae79-3d93643c7ffc None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:16.191139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f791ff-bb84-4ef1-ae79-3d93643c7ffc None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:16.191617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f791ff-bb84-4ef1-ae79-3d93643c7ffc None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:16.193659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 184/356] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:16.195275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f791ff-bb84-4ef1-ae79-3d93643c7ffc None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:16.345847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fdd449-e2df-47c5-ba48-5d262f0a80b9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:16.380265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f791ff-bb84-4ef1-ae79-3d93643c7ffc None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:16.382361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:16.413121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4ac1f-096e-4593-82d7-790d0c6df90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:16.414416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4ac1f-096e-4593-82d7-790d0c6df90e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:16.415234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4ac1f-096e-4593-82d7-790d0c6df90e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:16.415576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4ac1f-096e-4593-82d7-790d0c6df90e None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:16.425187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4ac1f-096e-4593-82d7-790d0c6df90e None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:16.427901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4ac1f-096e-4593-82d7-790d0c6df90e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:16.431089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 173/357] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:16 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:16.433264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4ac1f-096e-4593-82d7-790d0c6df90e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:16.442192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b51a6d-942b-41dc-bf5c-4f857007bd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:16.443103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51a6d-942b-41dc-bf5c-4f857007bd73 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:16.443548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51a6d-942b-41dc-bf5c-4f857007bd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:16.443961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b51a6d-942b-41dc-bf5c-4f857007bd73 None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:16.446369 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b51a6d-942b-41dc-bf5c-4f857007bd73 None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:16.448125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b51a6d-942b-41dc-bf5c-4f857007bd73 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:16.450019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 185/358] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:16 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:16.451211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b51a6d-942b-41dc-bf5c-4f857007bd73 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:16.540248 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:16.605254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b51a6d-942b-41dc-bf5c-4f857007bd73 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:16.615032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4ac1f-096e-4593-82d7-790d0c6df90e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:16.665093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651d148-a8ff-4557-b282-a2e26f7e67f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:16.665928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d148-a8ff-4557-b282-a2e26f7e67f3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:16.666211 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d148-a8ff-4557-b282-a2e26f7e67f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:16.666488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d148-a8ff-4557-b282-a2e26f7e67f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:16.671415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76e2c9-8ef4-427d-a78c-f6d5c0371d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:16.672499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76e2c9-8ef4-427d-a78c-f6d5c0371d8a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:16.672975 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76e2c9-8ef4-427d-a78c-f6d5c0371d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:16.673459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c76e2c9-8ef4-427d-a78c-f6d5c0371d8a None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:16.674828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76e2c9-8ef4-427d-a78c-f6d5c0371d8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:16.675359 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76e2c9-8ef4-427d-a78c-f6d5c0371d8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:16.678812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c76e2c9-8ef4-427d-a78c-f6d5c0371d8a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:16.682826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651d148-a8ff-4557-b282-a2e26f7e67f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d651d148-a8ff-4557-b282-a2e26f7e67f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:16.692192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:16.693139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:16.694021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:16.694733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 186/359] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:16.694805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 174/360] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:12:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:16.694875 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76e2c9-8ef4-427d-a78c-f6d5c0371d8a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:16.697923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f572c-1e4b-478c-b1c2-b1478b9faa74 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:16.698433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f572c-1e4b-478c-b1c2-b1478b9faa74 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:16.698830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f572c-1e4b-478c-b1c2-b1478b9faa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:16.719239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-cd4f572c-1e4b-478c-b1c2-b1478b9faa74 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:16.720693 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cd4f572c-1e4b-478c-b1c2-b1478b9faa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:16.742701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:16.843832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 175/361] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:16.845287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f572c-1e4b-478c-b1c2-b1478b9faa74 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:16.860255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c76e2c9-8ef4-427d-a78c-f6d5c0371d8a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:16.909335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca58fa-0551-47dd-ad0e-7d4a2e9183c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:16.911098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca58fa-0551-47dd-ad0e-7d4a2e9183c4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:16.911576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca58fa-0551-47dd-ad0e-7d4a2e9183c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:16.912026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca58fa-0551-47dd-ad0e-7d4a2e9183c4 None admin] PATH_INFO: `/v3/users/095170e111c24ffebab6232fedd8b26b` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:16.913884 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca58fa-0551-47dd-ad0e-7d4a2e9183c4 None admin] RBAC: Authorizing `identity:get_user(user_id=095170e111c24ffebab6232fedd8b26b)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:16.916704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca58fa-0551-47dd-ad0e-7d4a2e9183c4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:16.919601 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 187/362] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:12:16 2022] GET /identity/v3/users/095170e111c24ffebab6232fedd8b26b => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:16.920698 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca58fa-0551-47dd-ad0e-7d4a2e9183c4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:16.926292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:17.005624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f572c-1e4b-478c-b1c2-b1478b9faa74 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:17.046008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:17.049456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca58fa-0551-47dd-ad0e-7d4a2e9183c4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:17.052257 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a2295-8c0f-4939-acab-dd5df76fb4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:17.052926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a2295-8c0f-4939-acab-dd5df76fb4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:17.053205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a2295-8c0f-4939-acab-dd5df76fb4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:17.053479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a2295-8c0f-4939-acab-dd5df76fb4f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:17.054326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a2295-8c0f-4939-acab-dd5df76fb4f0 None admin] RBAC: Adding query filter params (group.id=f924c24d45a94a91be4b94e606ed767e, role.id=f4461b279e86422b83e29521e7ca29d0, scope.project.id=7139ad672c734cf799e4c39623384e9c) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:17.054660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a2295-8c0f-4939-acab-dd5df76fb4f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:17.056121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a2295-8c0f-4939-acab-dd5df76fb4f0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:17.064316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 176/363] 10.222.0.93 () {56 vars in 1376 bytes} [Fri Jun 3 15:12:16 2022] GET /identity/v3/role_assignments?group.id=f924c24d45a94a91be4b94e606ed767e&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=f4461b279e86422b83e29521e7ca29d0 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:17.064316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a2295-8c0f-4939-acab-dd5df76fb4f0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:17.101821 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94f242-d1ab-41c5-8a07-28c46faee5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:17.102489 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94f242-d1ab-41c5-8a07-28c46faee5da None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:17.102995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94f242-d1ab-41c5-8a07-28c46faee5da None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:17.103149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94f242-d1ab-41c5-8a07-28c46faee5da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:17.103965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94f242-d1ab-41c5-8a07-28c46faee5da None admin] RBAC: Adding query filter params (group.id=f924c24d45a94a91be4b94e606ed767e, role.id=1249af5cf5054410a11b2d88c03fcd52, scope.project.id=7139ad672c734cf799e4c39623384e9c) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:17.104274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94f242-d1ab-41c5-8a07-28c46faee5da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:17.105767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94f242-d1ab-41c5-8a07-28c46faee5da None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:17.123129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 188/364] 10.222.0.93 () {56 vars in 1376 bytes} [Fri Jun 3 15:12:16 2022] GET /identity/v3/role_assignments?group.id=f924c24d45a94a91be4b94e606ed767e&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:17.123129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94f242-d1ab-41c5-8a07-28c46faee5da None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:17.238687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a2295-8c0f-4939-acab-dd5df76fb4f0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:17.250115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:17.280797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4842f6d-1023-4581-b9af-89c51ab8fc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:17.281503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4842f6d-1023-4581-b9af-89c51ab8fc7d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:17.281791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4842f6d-1023-4581-b9af-89c51ab8fc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:17.282071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4842f6d-1023-4581-b9af-89c51ab8fc7d None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:17.282900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4842f6d-1023-4581-b9af-89c51ab8fc7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:17.283214 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4842f6d-1023-4581-b9af-89c51ab8fc7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:17.284672 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4842f6d-1023-4581-b9af-89c51ab8fc7d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:17.296862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 177/365] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:17.297773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4842f6d-1023-4581-b9af-89c51ab8fc7d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:17.324094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94f242-d1ab-41c5-8a07-28c46faee5da None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:17.372844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29b31d-326f-4da0-8b3c-fa976b3d3071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:17.373615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29b31d-326f-4da0-8b3c-fa976b3d3071 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:17.373793 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29b31d-326f-4da0-8b3c-fa976b3d3071 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:17.374148 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29b31d-326f-4da0-8b3c-fa976b3d3071 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:17.413816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:17.477791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4842f6d-1023-4581-b9af-89c51ab8fc7d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:17.490772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29b31d-326f-4da0-8b3c-fa976b3d3071 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:17.492642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29b31d-326f-4da0-8b3c-fa976b3d3071 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:17.536474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba36322-dfc4-4d2e-a867-d85b88b09741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:17.537662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba36322-dfc4-4d2e-a867-d85b88b09741 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:17.538071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba36322-dfc4-4d2e-a867-d85b88b09741 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:17.538471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba36322-dfc4-4d2e-a867-d85b88b09741 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:17.556581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba36322-dfc4-4d2e-a867-d85b88b09741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9ba36322-dfc4-4d2e-a867-d85b88b09741 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:17.570292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:17.572029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:12:17.573596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:17.578933 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 178/366] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:12:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:17.580540 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba36322-dfc4-4d2e-a867-d85b88b09741 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:17.620203 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:17.690387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 189/367] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:17.692159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29b31d-326f-4da0-8b3c-fa976b3d3071 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:17.707911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba36322-dfc4-4d2e-a867-d85b88b09741 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:17.747513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b0777-80ed-4645-8ef1-4fd0c7ce0e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:17.748229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0777-80ed-4645-8ef1-4fd0c7ce0e82 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:17.748906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0777-80ed-4645-8ef1-4fd0c7ce0e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:17.748906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0777-80ed-4645-8ef1-4fd0c7ce0e82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:17.749649 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b0777-80ed-4645-8ef1-4fd0c7ce0e82 None admin] RBAC: Adding query filter params (role.id=989320df8c4149cf918861e3ee5954d2, user.id=095170e111c24ffebab6232fedd8b26b, scope.system=all) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:17.749962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b0777-80ed-4645-8ef1-4fd0c7ce0e82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:17.751760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b0777-80ed-4645-8ef1-4fd0c7ce0e82 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:17.774563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 179/368] 10.222.0.93 () {56 vars in 1308 bytes} [Fri Jun 3 15:12:17 2022] GET /identity/v3/role_assignments?user.id=095170e111c24ffebab6232fedd8b26b&scope.system=all&role.id=989320df8c4149cf918861e3ee5954d2 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:17.774563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b0777-80ed-4645-8ef1-4fd0c7ce0e82 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:17.812084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:17.927284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:17.933892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b0777-80ed-4645-8ef1-4fd0c7ce0e82 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:17.941947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29b31d-326f-4da0-8b3c-fa976b3d3071 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:18.031269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016cdd7-aa72-4088-84d7-51a106e598e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:18.032084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016cdd7-aa72-4088-84d7-51a106e598e2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:18.032438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016cdd7-aa72-4088-84d7-51a106e598e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:18.032722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c016cdd7-aa72-4088-84d7-51a106e598e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:18.033489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016cdd7-aa72-4088-84d7-51a106e598e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:18.033797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016cdd7-aa72-4088-84d7-51a106e598e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:18.035283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c016cdd7-aa72-4088-84d7-51a106e598e2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:18.047755 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 180/369] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:18.049336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016cdd7-aa72-4088-84d7-51a106e598e2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:18.055049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435395b-c3d0-4080-8974-0069d69a829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:18.056055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435395b-c3d0-4080-8974-0069d69a829d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:18.056537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435395b-c3d0-4080-8974-0069d69a829d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:18.057014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435395b-c3d0-4080-8974-0069d69a829d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:18.186510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435395b-c3d0-4080-8974-0069d69a829d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:18.188071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435395b-c3d0-4080-8974-0069d69a829d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:18.229285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c016cdd7-aa72-4088-84d7-51a106e598e2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:18.261533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:18.278062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfef8d1-2fea-4c64-853d-f3cb12571a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:18.279561 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfef8d1-2fea-4c64-853d-f3cb12571a70 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:18.279856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfef8d1-2fea-4c64-853d-f3cb12571a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:18.280185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfef8d1-2fea-4c64-853d-f3cb12571a70 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:18.287213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfef8d1-2fea-4c64-853d-f3cb12571a70 None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:18.288958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfef8d1-2fea-4c64-853d-f3cb12571a70 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:18.292183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 181/370] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:18 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:18.292183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfef8d1-2fea-4c64-853d-f3cb12571a70 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:18.308226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 190/371] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:18.308872 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435395b-c3d0-4080-8974-0069d69a829d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:18.341408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:18.507810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435395b-c3d0-4080-8974-0069d69a829d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:18.511526 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfef8d1-2fea-4c64-853d-f3cb12571a70 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:18.580209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ce116-b06a-4726-8864-f779d9836b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:18.580935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ce116-b06a-4726-8864-f779d9836b06 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:18.581208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ce116-b06a-4726-8864-f779d9836b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:18.581494 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ce116-b06a-4726-8864-f779d9836b06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:18.596755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a75d62-13fa-43ff-8d70-5c1c591192c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:18.597960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75d62-13fa-43ff-8d70-5c1c591192c9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:18.598387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75d62-13fa-43ff-8d70-5c1c591192c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:18.598835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a75d62-13fa-43ff-8d70-5c1c591192c9 None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:18.601769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a75d62-13fa-43ff-8d70-5c1c591192c9 None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:18.605077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a75d62-13fa-43ff-8d70-5c1c591192c9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:18.610084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 191/372] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:18 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:18.613472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a75d62-13fa-43ff-8d70-5c1c591192c9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:18.695823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:18.710944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ce116-b06a-4726-8864-f779d9836b06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:18.712578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ce116-b06a-4726-8864-f779d9836b06 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:18.739483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:18.788903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a75d62-13fa-43ff-8d70-5c1c591192c9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:18.836301 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d2270-d0fa-46d4-a1d0-074c8cee171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:18.837636 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d2270-d0fa-46d4-a1d0-074c8cee171c None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:18.838180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d2270-d0fa-46d4-a1d0-074c8cee171c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:18.839822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d2270-d0fa-46d4-a1d0-074c8cee171c None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:18.842160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d2270-d0fa-46d4-a1d0-074c8cee171c None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:18.843968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d2270-d0fa-46d4-a1d0-074c8cee171c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:18.850879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 192/373] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:18 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:18.852069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d2270-d0fa-46d4-a1d0-074c8cee171c None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:18.869059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 182/374] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:18.870716 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ce116-b06a-4726-8864-f779d9836b06 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:19.022202 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:19.027489 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d2270-d0fa-46d4-a1d0-074c8cee171c None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:19.073398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ce116-b06a-4726-8864-f779d9836b06 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:19.077143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87a0c1-7c74-4093-9602-436de0786981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:19.078943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87a0c1-7c74-4093-9602-436de0786981 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:19.078943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87a0c1-7c74-4093-9602-436de0786981 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:19.078943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87a0c1-7c74-4093-9602-436de0786981 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:19.079580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87a0c1-7c74-4093-9602-436de0786981 None admin] RBAC: Adding query filter params (group.id=f924c24d45a94a91be4b94e606ed767e, role.id=1249af5cf5054410a11b2d88c03fcd52, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:19.080023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87a0c1-7c74-4093-9602-436de0786981 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:19.081655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87a0c1-7c74-4093-9602-436de0786981 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:19.087847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 193/375] 10.222.0.93 () {56 vars in 1376 bytes} [Fri Jun 3 15:12:18 2022] GET /identity/v3/role_assignments?group.id=f924c24d45a94a91be4b94e606ed767e&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:19.088806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87a0c1-7c74-4093-9602-436de0786981 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:19.102900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:19.131065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e7cbd-ed11-4027-81fe-b491d6e20bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:19.132229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e7cbd-ed11-4027-81fe-b491d6e20bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:19.133242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e7cbd-ed11-4027-81fe-b491d6e20bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:19.133670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e7cbd-ed11-4027-81fe-b491d6e20bc4 None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:19.135505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e7cbd-ed11-4027-81fe-b491d6e20bc4 None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:19.137281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e7cbd-ed11-4027-81fe-b491d6e20bc4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:19.139449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 183/376] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:18 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:19.146713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e7cbd-ed11-4027-81fe-b491d6e20bc4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:19.322375 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e7cbd-ed11-4027-81fe-b491d6e20bc4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:19.377354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2993614-1204-4c7d-a1e6-2e123ba48f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:19.377977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2993614-1204-4c7d-a1e6-2e123ba48f58 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:19.378263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2993614-1204-4c7d-a1e6-2e123ba48f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:19.378670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2993614-1204-4c7d-a1e6-2e123ba48f58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:19.379673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2993614-1204-4c7d-a1e6-2e123ba48f58 None admin] RBAC: Adding query filter params (group.id=f924c24d45a94a91be4b94e606ed767e, role.id=f4461b279e86422b83e29521e7ca29d0, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:19.380057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2993614-1204-4c7d-a1e6-2e123ba48f58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:19.381544 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2993614-1204-4c7d-a1e6-2e123ba48f58 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:19.391045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 184/377] 10.222.0.93 () {56 vars in 1376 bytes} [Fri Jun 3 15:12:19 2022] GET /identity/v3/role_assignments?group.id=f924c24d45a94a91be4b94e606ed767e&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=f4461b279e86422b83e29521e7ca29d0 => generated 260 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:19.417659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87a0c1-7c74-4093-9602-436de0786981 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:19.474639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea80113-79c1-4536-b967-cca820c9a712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:19.476093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea80113-79c1-4536-b967-cca820c9a712 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:19.476572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea80113-79c1-4536-b967-cca820c9a712 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:19.477007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea80113-79c1-4536-b967-cca820c9a712 None admin] PATH_INFO: `/v3/projects/4b7e9c80374f4695bdf655157e8cb4d3` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:19.479186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea80113-79c1-4536-b967-cca820c9a712 None admin] RBAC: Authorizing `identity:get_project(project_id=4b7e9c80374f4695bdf655157e8cb4d3)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:19.481868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea80113-79c1-4536-b967-cca820c9a712 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:19.484902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 194/378] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:19 2022] GET /identity/v3/projects/4b7e9c80374f4695bdf655157e8cb4d3 => generated 344 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:19.490841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2993614-1204-4c7d-a1e6-2e123ba48f58 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:19.592785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:19.697772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2993614-1204-4c7d-a1e6-2e123ba48f58 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:19.769701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab776d6-04dd-4cb8-8050-9865bd46164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:19.770915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab776d6-04dd-4cb8-8050-9865bd46164f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:19.771379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab776d6-04dd-4cb8-8050-9865bd46164f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:19.771823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab776d6-04dd-4cb8-8050-9865bd46164f None admin] PATH_INFO: `/v3/groups/663850abe4df4cf287639abea09b0ef4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:19.779500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab776d6-04dd-4cb8-8050-9865bd46164f None admin] RBAC: Authorizing `identity:get_group(group_id=663850abe4df4cf287639abea09b0ef4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:19.781658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab776d6-04dd-4cb8-8050-9865bd46164f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:19.784072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 185/379] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:19 2022] GET /identity/v3/groups/663850abe4df4cf287639abea09b0ef4 => generated 229 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:19.792813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea80113-79c1-4536-b967-cca820c9a712 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:19.844537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:19.973862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea80113-79c1-4536-b967-cca820c9a712 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:19.982100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cd3e8-2127-4958-95a4-6890daaea2ea None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:19.982645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cd3e8-2127-4958-95a4-6890daaea2ea None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:19.983076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448cd3e8-2127-4958-95a4-6890daaea2ea None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:19.988840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 186/380] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:19.990745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c8fa7-b55f-40a8-a2e6-c7e4c5a7ff98 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:19.991038 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c8fa7-b55f-40a8-a2e6-c7e4c5a7ff98 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:19.991322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c8fa7-b55f-40a8-a2e6-c7e4c5a7ff98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:20.019195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ac7c8fa7-b55f-40a8-a2e6-c7e4c5a7ff98 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:20.021855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ac7c8fa7-b55f-40a8-a2e6-c7e4c5a7ff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:20.045900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234dd052-e64c-461f-86d5-2929aaf98649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:20.046753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dd052-e64c-461f-86d5-2929aaf98649 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:20.046890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dd052-e64c-461f-86d5-2929aaf98649 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:20.047270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dd052-e64c-461f-86d5-2929aaf98649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:20.047993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234dd052-e64c-461f-86d5-2929aaf98649 None admin] RBAC: Adding query filter params (group.id=663850abe4df4cf287639abea09b0ef4, role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=4b7e9c80374f4695bdf655157e8cb4d3) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:20.048385 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234dd052-e64c-461f-86d5-2929aaf98649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:20.050249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234dd052-e64c-461f-86d5-2929aaf98649 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:20.062447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 195/381] 10.222.0.93 () {56 vars in 1376 bytes} [Fri Jun 3 15:12:19 2022] GET /identity/v3/role_assignments?group.id=663850abe4df4cf287639abea09b0ef4&scope.project.id=4b7e9c80374f4695bdf655157e8cb4d3&role.id=d67707782358417ba9f25b2793ca2890 => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:20.182749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 187/382] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:20.223084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234dd052-e64c-461f-86d5-2929aaf98649 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:20.286565 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d8a1c-96ac-48d7-bc51-3fcece4285b8 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:20.287023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d8a1c-96ac-48d7-bc51-3fcece4285b8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:20.287467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d8a1c-96ac-48d7-bc51-3fcece4285b8 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:20.290883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 188/383] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:20.297272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2cab0-f17d-4de2-8e40-16a06c5dc3cb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:20.298009 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2cab0-f17d-4de2-8e40-16a06c5dc3cb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:20.298700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2cab0-f17d-4de2-8e40-16a06c5dc3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:20.334235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b4a2cab0-f17d-4de2-8e40-16a06c5dc3cb None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:20.337471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b4a2cab0-f17d-4de2-8e40-16a06c5dc3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:20.410934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:20.445460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:20.451348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 189/384] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:20.471457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234dd052-e64c-461f-86d5-2929aaf98649 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:20.476646 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2cab0-f17d-4de2-8e40-16a06c5dc3cb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:20.553101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d2075-c1a7-44b6-9ce1-b77fa05874c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:20.553815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d2075-c1a7-44b6-9ce1-b77fa05874c0 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:20.554102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d2075-c1a7-44b6-9ce1-b77fa05874c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:20.554373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d2075-c1a7-44b6-9ce1-b77fa05874c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:20.568355 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d2075-c1a7-44b6-9ce1-b77fa05874c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-9c8d2075-c1a7-44b6-9ce1-b77fa05874c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:20.576325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:20.577645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:20.579004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 196/385] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:20.587431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d2075-c1a7-44b6-9ce1-b77fa05874c0 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:20.668414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2cab0-f17d-4de2-8e40-16a06c5dc3cb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:20.727128 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c1b45-d848-497b-a7ec-759d1e764edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:20.728361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c1b45-d848-497b-a7ec-759d1e764edc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:20.728850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c1b45-d848-497b-a7ec-759d1e764edc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:20.729340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c1b45-d848-497b-a7ec-759d1e764edc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:20.748511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c1b45-d848-497b-a7ec-759d1e764edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ca6c1b45-d848-497b-a7ec-759d1e764edc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:20.766133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:20.767139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:20.767924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:20.770819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 190/386] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:12:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:20.781627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c1b45-d848-497b-a7ec-759d1e764edc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:20.806216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d2075-c1a7-44b6-9ce1-b77fa05874c0 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:20.852649 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:20.858089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d31966-20d2-4128-904c-92e508495f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:20.858881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d31966-20d2-4128-904c-92e508495f66 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:20.859178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d31966-20d2-4128-904c-92e508495f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:20.859442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d31966-20d2-4128-904c-92e508495f66 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:20.860246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d31966-20d2-4128-904c-92e508495f66 None admin] RBAC: Adding query filter params (name=member) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:20.860611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d31966-20d2-4128-904c-92e508495f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:20.862064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d31966-20d2-4128-904c-92e508495f66 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:20.871714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 197/387] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:20.877659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d31966-20d2-4128-904c-92e508495f66 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:20.922080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:20.952304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c1b45-d848-497b-a7ec-759d1e764edc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:21.011480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393d91b-77a6-42ad-b99a-83cbfe9701ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:21.013321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d91b-77a6-42ad-b99a-83cbfe9701ab None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:21.014762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d91b-77a6-42ad-b99a-83cbfe9701ab None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:21.015430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393d91b-77a6-42ad-b99a-83cbfe9701ab None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:21.018701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393d91b-77a6-42ad-b99a-83cbfe9701ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:21.019689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393d91b-77a6-42ad-b99a-83cbfe9701ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:21.022795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393d91b-77a6-42ad-b99a-83cbfe9701ab None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:21.043728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 191/388] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:21.052465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393d91b-77a6-42ad-b99a-83cbfe9701ab None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:21.099667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d31966-20d2-4128-904c-92e508495f66 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:21.152615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ab06c-c21b-42d7-98e8-e09767c9d787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:21.153547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ab06c-c21b-42d7-98e8-e09767c9d787 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:21.153834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ab06c-c21b-42d7-98e8-e09767c9d787 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:21.154112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ab06c-c21b-42d7-98e8-e09767c9d787 None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:21.159404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ab06c-c21b-42d7-98e8-e09767c9d787 None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:21.160917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ab06c-c21b-42d7-98e8-e09767c9d787 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:21.163054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 198/389] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:20 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:21.168733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ab06c-c21b-42d7-98e8-e09767c9d787 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:21.249658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:21.262333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393d91b-77a6-42ad-b99a-83cbfe9701ab None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:21.316324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fa9ac-9e0a-4898-8430-0903be223fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:21.317496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fa9ac-9e0a-4898-8430-0903be223fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:21.317984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fa9ac-9e0a-4898-8430-0903be223fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:21.318429 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194fa9ac-9e0a-4898-8430-0903be223fe9 None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:21.323313 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fa9ac-9e0a-4898-8430-0903be223fe9 None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:21.325627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194fa9ac-9e0a-4898-8430-0903be223fe9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:21.327873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 192/390] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:21 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:21.336518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fa9ac-9e0a-4898-8430-0903be223fe9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:21.340438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:21.373831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ab06c-c21b-42d7-98e8-e09767c9d787 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:21.451885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63cecf-4ff0-4e55-9455-82f4882017bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:21.453145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63cecf-4ff0-4e55-9455-82f4882017bf None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:21.457406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63cecf-4ff0-4e55-9455-82f4882017bf None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:21.458041 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63cecf-4ff0-4e55-9455-82f4882017bf None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:21.459684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63cecf-4ff0-4e55-9455-82f4882017bf None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:21.461401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad63cecf-4ff0-4e55-9455-82f4882017bf None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:21.465805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 199/391] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:21 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:21.474049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63cecf-4ff0-4e55-9455-82f4882017bf None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:21.567245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194fa9ac-9e0a-4898-8430-0903be223fe9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:21.664140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63cecf-4ff0-4e55-9455-82f4882017bf None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:21.693290 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbdf7a-e330-4ee6-a0db-ef36334a90cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:21.694559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbdf7a-e330-4ee6-a0db-ef36334a90cd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:21.696517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbdf7a-e330-4ee6-a0db-ef36334a90cd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:21.699693 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbdf7a-e330-4ee6-a0db-ef36334a90cd None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:21.701714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbdf7a-e330-4ee6-a0db-ef36334a90cd None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:21.703399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbdf7a-e330-4ee6-a0db-ef36334a90cd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:21.715225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 193/392] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:21 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:21.730988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff0f4c-e13f-4014-96cb-873fce993114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:21.732705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbdf7a-e330-4ee6-a0db-ef36334a90cd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:21.735637 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff0f4c-e13f-4014-96cb-873fce993114 None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:21.736057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff0f4c-e13f-4014-96cb-873fce993114 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:21.736439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff0f4c-e13f-4014-96cb-873fce993114 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c/groups/f924c24d45a94a91be4b94e606ed767e/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:21.742115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff0f4c-e13f-4014-96cb-873fce993114 None admin] RBAC: Authorizing `identity:create_grant(project_id=7139ad672c734cf799e4c39623384e9c, group_id=f924c24d45a94a91be4b94e606ed767e, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:21.743791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff0f4c-e13f-4014-96cb-873fce993114 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:21.766442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 200/393] 10.222.0.93 () {58 vars in 1442 bytes} [Fri Jun 3 15:12:21 2022] PUT /identity/v3/projects/7139ad672c734cf799e4c39623384e9c/groups/f924c24d45a94a91be4b94e606ed767e/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:21.797665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:21.908162 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbdf7a-e330-4ee6-a0db-ef36334a90cd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:21.964017 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81a051-877f-44f9-92a8-174c7345c5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:21.965238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81a051-877f-44f9-92a8-174c7345c5d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:21.965730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81a051-877f-44f9-92a8-174c7345c5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:21.966192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81a051-877f-44f9-92a8-174c7345c5d4 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c/groups/f924c24d45a94a91be4b94e606ed767e/roles/f4461b279e86422b83e29521e7ca29d0` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:21.968855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81a051-877f-44f9-92a8-174c7345c5d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7139ad672c734cf799e4c39623384e9c, group_id=f924c24d45a94a91be4b94e606ed767e, role_id=f4461b279e86422b83e29521e7ca29d0)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:21.970611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81a051-877f-44f9-92a8-174c7345c5d4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:21.990308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 194/394] 10.222.0.93 () {58 vars in 1442 bytes} [Fri Jun 3 15:12:21 2022] PUT /identity/v3/projects/7139ad672c734cf799e4c39623384e9c/groups/f924c24d45a94a91be4b94e606ed767e/roles/f4461b279e86422b83e29521e7ca29d0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:22.584550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ede38e-b434-4a8d-b461-fb0efe358aef None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:22.585073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ede38e-b434-4a8d-b461-fb0efe358aef None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:22.585073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ede38e-b434-4a8d-b461-fb0efe358aef None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:22.586778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 201/395] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:22.592640 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e95ca-630f-4901-83ce-c2d5b6484fc9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:22.593681 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e95ca-630f-4901-83ce-c2d5b6484fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:22.594451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e95ca-630f-4901-83ce-c2d5b6484fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:22.620485 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-176e95ca-630f-4901-83ce-c2d5b6484fc9 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:22.622872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-176e95ca-630f-4901-83ce-c2d5b6484fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:22.755088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 195/396] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:22.792705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ede38e-b434-4a8d-b461-fb0efe358aef None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:22.813615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:22.898103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff59b3-f3be-4edb-8852-9f7a3f7b9650 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:22.898405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff59b3-f3be-4edb-8852-9f7a3f7b9650 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:22.898710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff59b3-f3be-4edb-8852-9f7a3f7b9650 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:22.900208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 196/397] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:22.910076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cd6dc-77b1-4afa-a5c2-4177905900db None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:22.910490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cd6dc-77b1-4afa-a5c2-4177905900db None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:22.910676 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cd6dc-77b1-4afa-a5c2-4177905900db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:22.956413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5b4cd6dc-77b1-4afa-a5c2-4177905900db None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:22.958308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5b4cd6dc-77b1-4afa-a5c2-4177905900db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:22.964522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ede38e-b434-4a8d-b461-fb0efe358aef None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:22.995080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:23.051289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b10987-ea43-4372-b6de-b48e0bfcedc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:23.052105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b10987-ea43-4372-b6de-b48e0bfcedc9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:23.052382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b10987-ea43-4372-b6de-b48e0bfcedc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:23.052652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b10987-ea43-4372-b6de-b48e0bfcedc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:23.066180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b10987-ea43-4372-b6de-b48e0bfcedc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-30b10987-ea43-4372-b6de-b48e0bfcedc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:23.080886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:23.081891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:23.082740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:23.089304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 202/398] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:23.090685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 197/399] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:12:23.098586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b10987-ea43-4372-b6de-b48e0bfcedc9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:23.152824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cd6dc-77b1-4afa-a5c2-4177905900db None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:23.237014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:23.285010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b10987-ea43-4372-b6de-b48e0bfcedc9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:23.337272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5c960-dc28-4aab-bf71-8812f6e49490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:23.338401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5c960-dc28-4aab-bf71-8812f6e49490 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:23.339186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5c960-dc28-4aab-bf71-8812f6e49490 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:23.339650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5c960-dc28-4aab-bf71-8812f6e49490 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:23.342577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5c960-dc28-4aab-bf71-8812f6e49490 None admin] RBAC: Adding query filter params (name=member) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:23.343765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5c960-dc28-4aab-bf71-8812f6e49490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:23.347325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5c960-dc28-4aab-bf71-8812f6e49490 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:23.361353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 203/400] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:23.363602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b0aff-083c-43e7-9b41-e851805d4871 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:23.364508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b0aff-083c-43e7-9b41-e851805d4871 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:23.364976 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b0aff-083c-43e7-9b41-e851805d4871 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:23.366962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 204/401] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:23.371680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b0aff-083c-43e7-9b41-e851805d4871 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:23.390433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cd6dc-77b1-4afa-a5c2-4177905900db None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:23.442411 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46df92f6-4afd-4cb1-9ea6-3e6878de1145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:23.443801 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df92f6-4afd-4cb1-9ea6-3e6878de1145 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:23.444372 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df92f6-4afd-4cb1-9ea6-3e6878de1145 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:23.445027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46df92f6-4afd-4cb1-9ea6-3e6878de1145 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:23.461104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46df92f6-4afd-4cb1-9ea6-3e6878de1145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-46df92f6-4afd-4cb1-9ea6-3e6878de1145 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:23.470240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:23.471374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:23.472179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:23.474520 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 198/402] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:12:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:23.476021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e6868-f765-439d-8b81-23214863889a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:23.476512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e6868-f765-439d-8b81-23214863889a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:23.477045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e6868-f765-439d-8b81-23214863889a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:23.502369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-690e6868-f765-439d-8b81-23214863889a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:23.504040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-690e6868-f765-439d-8b81-23214863889a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:23.513003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:23.532768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b0aff-083c-43e7-9b41-e851805d4871 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:23.580985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f0d45-977d-422e-9f3a-0cfcb1c45e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:23.581894 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f0d45-977d-422e-9f3a-0cfcb1c45e19 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:23.582209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f0d45-977d-422e-9f3a-0cfcb1c45e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:23.582518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f0d45-977d-422e-9f3a-0cfcb1c45e19 None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:23.583892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f0d45-977d-422e-9f3a-0cfcb1c45e19 None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:23.585411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f0d45-977d-422e-9f3a-0cfcb1c45e19 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:23.590748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 205/403] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:23 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:23.592589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f0d45-977d-422e-9f3a-0cfcb1c45e19 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:23.603926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:23.626650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 199/404] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:23.628565 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e6868-f765-439d-8b81-23214863889a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:23.751154 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f0d45-977d-422e-9f3a-0cfcb1c45e19 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:23.799819 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a6a82-b594-4c5e-80e5-1d11e1d12fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:23.801308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a6a82-b594-4c5e-80e5-1d11e1d12fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:23.801657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a6a82-b594-4c5e-80e5-1d11e1d12fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:23.801942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6a6a82-b594-4c5e-80e5-1d11e1d12fa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:23.804013 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a6a82-b594-4c5e-80e5-1d11e1d12fa2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:23.804520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a6a82-b594-4c5e-80e5-1d11e1d12fa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:23.807300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6a6a82-b594-4c5e-80e5-1d11e1d12fa2 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:23.822225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e6868-f765-439d-8b81-23214863889a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:23.823999 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 206/405] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:23.825964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a6a82-b594-4c5e-80e5-1d11e1d12fa2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:23.918726 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bafdba-e2d7-490c-bcce-0206b1eb2154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:23.920523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bafdba-e2d7-490c-bcce-0206b1eb2154 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:23.920868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bafdba-e2d7-490c-bcce-0206b1eb2154 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:23.921250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bafdba-e2d7-490c-bcce-0206b1eb2154 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:23.927713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bafdba-e2d7-490c-bcce-0206b1eb2154 None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:23.930804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bafdba-e2d7-490c-bcce-0206b1eb2154 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:23.935673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 200/406] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:23 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:23.937117 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bafdba-e2d7-490c-bcce-0206b1eb2154 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:23.959684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:23.969486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:24.011536 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6a6a82-b594-4c5e-80e5-1d11e1d12fa2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:24.062518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6e23e-709b-4dcf-b475-3082d7c541cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:24.063365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6e23e-709b-4dcf-b475-3082d7c541cc None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:24.063650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6e23e-709b-4dcf-b475-3082d7c541cc None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:24.063992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6e23e-709b-4dcf-b475-3082d7c541cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:24.079611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6e23e-709b-4dcf-b475-3082d7c541cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:24.085912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bafdba-e2d7-490c-bcce-0206b1eb2154 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-74e6e23e-709b-4dcf-b475-3082d7c541cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:24.087519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:24.088919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:12:24.090323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:24.094334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 207/407] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:12:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:24.095631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6e23e-709b-4dcf-b475-3082d7c541cc None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:24.149638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5595714-5426-4048-9343-3b44d8d58179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:24.151426 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5595714-5426-4048-9343-3b44d8d58179 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:24.152236 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5595714-5426-4048-9343-3b44d8d58179 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:24.152934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5595714-5426-4048-9343-3b44d8d58179 None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:24.155656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5595714-5426-4048-9343-3b44d8d58179 None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:24.159033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5595714-5426-4048-9343-3b44d8d58179 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:24.162382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 201/408] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:23 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:24.164014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5595714-5426-4048-9343-3b44d8d58179 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:24.227621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6e23e-709b-4dcf-b475-3082d7c541cc None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:24.257449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:24.268830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904167d7-2a33-4922-8bf1-8f51e8bb26cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:24.269644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904167d7-2a33-4922-8bf1-8f51e8bb26cc None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:24.269928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904167d7-2a33-4922-8bf1-8f51e8bb26cc None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:24.270208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904167d7-2a33-4922-8bf1-8f51e8bb26cc None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/groups/f924c24d45a94a91be4b94e606ed767e/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:24.272613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904167d7-2a33-4922-8bf1-8f51e8bb26cc None admin] RBAC: Authorizing `identity:create_grant(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9, group_id=f924c24d45a94a91be4b94e606ed767e, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:24.273986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904167d7-2a33-4922-8bf1-8f51e8bb26cc None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:24.297878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:24.301399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 208/409] 10.222.0.93 () {58 vars in 1442 bytes} [Fri Jun 3 15:12:24 2022] PUT /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/groups/f924c24d45a94a91be4b94e606ed767e/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:24.307169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904167d7-2a33-4922-8bf1-8f51e8bb26cc None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:24.345191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5595714-5426-4048-9343-3b44d8d58179 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:24.394432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1c912-480e-43b6-bc15-fa8dfb2f287c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:24.395607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1c912-480e-43b6-bc15-fa8dfb2f287c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:24.396096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1c912-480e-43b6-bc15-fa8dfb2f287c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:24.396721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1c912-480e-43b6-bc15-fa8dfb2f287c None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:24.397750 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1c912-480e-43b6-bc15-fa8dfb2f287c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:24.398634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1c912-480e-43b6-bc15-fa8dfb2f287c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:24.400515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1c912-480e-43b6-bc15-fa8dfb2f287c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:24.415621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 202/410] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:24.425843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1c912-480e-43b6-bc15-fa8dfb2f287c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:24.602480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1c912-480e-43b6-bc15-fa8dfb2f287c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:24.632950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904167d7-2a33-4922-8bf1-8f51e8bb26cc None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:24.643647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8542f56-d475-47a3-9a81-a3b2743abb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:24.645598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542f56-d475-47a3-9a81-a3b2743abb84 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:24.645598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542f56-d475-47a3-9a81-a3b2743abb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:24.645598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542f56-d475-47a3-9a81-a3b2743abb84 None admin] PATH_INFO: `/v3/groups/663850abe4df4cf287639abea09b0ef4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:24.646338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8542f56-d475-47a3-9a81-a3b2743abb84 None admin] RBAC: Authorizing `identity:get_group(group_id=663850abe4df4cf287639abea09b0ef4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:24.648455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8542f56-d475-47a3-9a81-a3b2743abb84 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:24.660323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 203/411] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:24 2022] GET /identity/v3/groups/663850abe4df4cf287639abea09b0ef4 => generated 229 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:24.661299 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8542f56-d475-47a3-9a81-a3b2743abb84 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:24.679673 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ca644-cc51-49de-8e44-4b74ef47bcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:24.680801 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ca644-cc51-49de-8e44-4b74ef47bcfa None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:24.681304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ca644-cc51-49de-8e44-4b74ef47bcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:24.681832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ca644-cc51-49de-8e44-4b74ef47bcfa None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:24.683353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ca644-cc51-49de-8e44-4b74ef47bcfa None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:24.685077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ca644-cc51-49de-8e44-4b74ef47bcfa None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:24.685659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:24.690281 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 209/412] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:24 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:24.699242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ca644-cc51-49de-8e44-4b74ef47bcfa None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:24.808462 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:24.860015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8542f56-d475-47a3-9a81-a3b2743abb84 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:24.936327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728713f-eeed-4ac1-a755-13002f19b4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:24.937492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728713f-eeed-4ac1-a755-13002f19b4ac None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:24.937936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728713f-eeed-4ac1-a755-13002f19b4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:24.938293 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728713f-eeed-4ac1-a755-13002f19b4ac None admin] PATH_INFO: `/v3/projects/4b7e9c80374f4695bdf655157e8cb4d3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:24.941815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ca644-cc51-49de-8e44-4b74ef47bcfa None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:24.944694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728713f-eeed-4ac1-a755-13002f19b4ac None admin] RBAC: Authorizing `identity:get_project(project_id=4b7e9c80374f4695bdf655157e8cb4d3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:24.946810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728713f-eeed-4ac1-a755-13002f19b4ac None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:24.952459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 204/413] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:24 2022] GET /identity/v3/projects/4b7e9c80374f4695bdf655157e8cb4d3 => generated 344 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:24.961662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728713f-eeed-4ac1-a755-13002f19b4ac None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:25.058440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72feb2-cb06-4c3a-ac77-ac5ccd9d736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:25.060579 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72feb2-cb06-4c3a-ac77-ac5ccd9d736d None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:25.061067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72feb2-cb06-4c3a-ac77-ac5ccd9d736d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:25.061555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72feb2-cb06-4c3a-ac77-ac5ccd9d736d None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/groups/f924c24d45a94a91be4b94e606ed767e/roles/f4461b279e86422b83e29521e7ca29d0` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:25.069958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72feb2-cb06-4c3a-ac77-ac5ccd9d736d None admin] RBAC: Authorizing `identity:create_grant(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9, group_id=f924c24d45a94a91be4b94e606ed767e, role_id=f4461b279e86422b83e29521e7ca29d0)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:25.072747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72feb2-cb06-4c3a-ac77-ac5ccd9d736d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:25.116496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 210/414] 10.222.0.93 () {58 vars in 1442 bytes} [Fri Jun 3 15:12:24 2022] PUT /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9/groups/f924c24d45a94a91be4b94e606ed767e/roles/f4461b279e86422b83e29521e7ca29d0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:12:25.192874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:25.256064 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728713f-eeed-4ac1-a755-13002f19b4ac None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:25.329082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03570837-814a-41d5-ae94-8c5f240f2a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:25.331708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf906da-2c6f-435f-bbcb-8e15844da749 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:25.332204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf906da-2c6f-435f-bbcb-8e15844da749 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:25.332676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf906da-2c6f-435f-bbcb-8e15844da749 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:25.335276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 211/415] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:25.337051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03570837-814a-41d5-ae94-8c5f240f2a00 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:25.337544 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03570837-814a-41d5-ae94-8c5f240f2a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:25.338030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03570837-814a-41d5-ae94-8c5f240f2a00 None admin] PATH_INFO: `/v3/projects/4b7e9c80374f4695bdf655157e8cb4d3/groups/663850abe4df4cf287639abea09b0ef4/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:25.342357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03570837-814a-41d5-ae94-8c5f240f2a00 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b7e9c80374f4695bdf655157e8cb4d3, group_id=663850abe4df4cf287639abea09b0ef4, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:25.345112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03570837-814a-41d5-ae94-8c5f240f2a00 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:25.355297 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390fcc2-7183-4dd8-8f29-5c919babb871 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:25.355956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390fcc2-7183-4dd8-8f29-5c919babb871 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:25.356476 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390fcc2-7183-4dd8-8f29-5c919babb871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:25.388331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 205/416] 10.222.0.93 () {58 vars in 1442 bytes} [Fri Jun 3 15:12:24 2022] PUT /identity/v3/projects/4b7e9c80374f4695bdf655157e8cb4d3/groups/663850abe4df4cf287639abea09b0ef4/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:25.398388 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-8390fcc2-7183-4dd8-8f29-5c919babb871 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:25.400463 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8390fcc2-7183-4dd8-8f29-5c919babb871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:25.447267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf4a1a-421b-4c51-8707-71f0d79e194f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:25.447788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf4a1a-421b-4c51-8707-71f0d79e194f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:25.448276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf4a1a-421b-4c51-8707-71f0d79e194f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:25.456081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 206/417] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:25 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:25.464138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0083c-c0f6-44f9-b7e4-84a9ad3e2a03 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:25.464579 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0083c-c0f6-44f9-b7e4-84a9ad3e2a03 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:25.465050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0083c-c0f6-44f9-b7e4-84a9ad3e2a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:25.497932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e7e0083c-c0f6-44f9-b7e4-84a9ad3e2a03 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:25.499867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e7e0083c-c0f6-44f9-b7e4-84a9ad3e2a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:25.526716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 212/418] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:25.562008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390fcc2-7183-4dd8-8f29-5c919babb871 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:25.626674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 207/419] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:12:25.663218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:25.686427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0083c-c0f6-44f9-b7e4-84a9ad3e2a03 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:25.772849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:25.774614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390fcc2-7183-4dd8-8f29-5c919babb871 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:25.834704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0083c-c0f6-44f9-b7e4-84a9ad3e2a03 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:25.848315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518ec19-47d5-415f-95ae-4683c93678b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:25.849130 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518ec19-47d5-415f-95ae-4683c93678b9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:25.849429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518ec19-47d5-415f-95ae-4683c93678b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:25.849684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518ec19-47d5-415f-95ae-4683c93678b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:25.871206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9518ec19-47d5-415f-95ae-4683c93678b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-9518ec19-47d5-415f-95ae-4683c93678b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:25.884055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:25.885002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:25.885792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 213/420] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:25.892012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a4231-c9f3-45c2-ad6f-b7d862bdfabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:25.893339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a4231-c9f3-45c2-ad6f-b7d862bdfabc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:25.893812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a4231-c9f3-45c2-ad6f-b7d862bdfabc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:25.894301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a4231-c9f3-45c2-ad6f-b7d862bdfabc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:25.897440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518ec19-47d5-415f-95ae-4683c93678b9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:25.915011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5a4231-c9f3-45c2-ad6f-b7d862bdfabc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ea5a4231-c9f3-45c2-ad6f-b7d862bdfabc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:25.928525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:25.931928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:25.935137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 208/421] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:12:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:25.942555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a4231-c9f3-45c2-ad6f-b7d862bdfabc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:26.050365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518ec19-47d5-415f-95ae-4683c93678b9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:26.067017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:26.116933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0473d2-598e-4b30-87bf-9a00e258837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:26.117858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0473d2-598e-4b30-87bf-9a00e258837a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:26.118200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0473d2-598e-4b30-87bf-9a00e258837a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:26.118612 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0473d2-598e-4b30-87bf-9a00e258837a None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:26.119668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0473d2-598e-4b30-87bf-9a00e258837a None admin] RBAC: Adding query filter params (name=member) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:26.120067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0473d2-598e-4b30-87bf-9a00e258837a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:26.121967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0473d2-598e-4b30-87bf-9a00e258837a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:26.150568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 214/422] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:26.152270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0473d2-598e-4b30-87bf-9a00e258837a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:26.176686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5a4231-c9f3-45c2-ad6f-b7d862bdfabc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:26.231566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:26.254345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf7c86-6f88-4d26-81a3-e426f08c2667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:26.255604 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7c86-6f88-4d26-81a3-e426f08c2667 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:26.255604 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7c86-6f88-4d26-81a3-e426f08c2667 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:26.255792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7c86-6f88-4d26-81a3-e426f08c2667 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:26.256540 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf7c86-6f88-4d26-81a3-e426f08c2667 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:26.256857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf7c86-6f88-4d26-81a3-e426f08c2667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:26.258275 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf7c86-6f88-4d26-81a3-e426f08c2667 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:26.273298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 209/423] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:26.280332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf7c86-6f88-4d26-81a3-e426f08c2667 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:26.336872 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0473d2-598e-4b30-87bf-9a00e258837a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:26.414847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66f4ce-2e15-475d-9ab2-e64e17272161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:26.415732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66f4ce-2e15-475d-9ab2-e64e17272161 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:26.416091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66f4ce-2e15-475d-9ab2-e64e17272161 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:26.416451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66f4ce-2e15-475d-9ab2-e64e17272161 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:26.511989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:26.576365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf7c86-6f88-4d26-81a3-e426f08c2667 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:26.591714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66f4ce-2e15-475d-9ab2-e64e17272161 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:26.593511 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66f4ce-2e15-475d-9ab2-e64e17272161 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:26.668340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695d71-ba4a-47a5-b7c4-844ae655a95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:26.669890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695d71-ba4a-47a5-b7c4-844ae655a95e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:26.670270 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695d71-ba4a-47a5-b7c4-844ae655a95e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:26.670683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695d71-ba4a-47a5-b7c4-844ae655a95e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:26.773025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 215/424] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:26.773814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:26.781328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66f4ce-2e15-475d-9ab2-e64e17272161 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:26.793596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0695d71-ba4a-47a5-b7c4-844ae655a95e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:26.796309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0695d71-ba4a-47a5-b7c4-844ae655a95e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:26.967087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:27.015308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 210/425] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:27.021080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695d71-ba4a-47a5-b7c4-844ae655a95e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:27.084325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66f4ce-2e15-475d-9ab2-e64e17272161 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:27.171736 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0839c4-bdd8-41fa-a65c-7b3b03461263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:27.173306 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0839c4-bdd8-41fa-a65c-7b3b03461263 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:27.173803 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0839c4-bdd8-41fa-a65c-7b3b03461263 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:27.174369 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0839c4-bdd8-41fa-a65c-7b3b03461263 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:27.180027 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0839c4-bdd8-41fa-a65c-7b3b03461263 None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:27.182788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0839c4-bdd8-41fa-a65c-7b3b03461263 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:27.195266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 216/426] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:26 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:27.202263 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0839c4-bdd8-41fa-a65c-7b3b03461263 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:27.226194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695d71-ba4a-47a5-b7c4-844ae655a95e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:27.247154 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:27.269401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb72b9-69de-4110-95d3-5e4855c0e061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:27.270430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb72b9-69de-4110-95d3-5e4855c0e061 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:27.270774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb72b9-69de-4110-95d3-5e4855c0e061 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:27.271238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb72b9-69de-4110-95d3-5e4855c0e061 None admin] PATH_INFO: `/v3/projects/7139ad672c734cf799e4c39623384e9c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:27.272546 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb72b9-69de-4110-95d3-5e4855c0e061 None admin] RBAC: Authorizing `identity:get_project(project_id=7139ad672c734cf799e4c39623384e9c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:27.274896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb72b9-69de-4110-95d3-5e4855c0e061 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:27.277358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 211/427] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:27 2022] GET /identity/v3/projects/7139ad672c734cf799e4c39623384e9c => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:27.289787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb72b9-69de-4110-95d3-5e4855c0e061 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:27.405419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb72b9-69de-4110-95d3-5e4855c0e061 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:27.446786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc46e5-b5f0-43fe-ae57-308775a600c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:27.447663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc46e5-b5f0-43fe-ae57-308775a600c2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:27.447949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc46e5-b5f0-43fe-ae57-308775a600c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:27.448252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc46e5-b5f0-43fe-ae57-308775a600c2 None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:27.449542 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc46e5-b5f0-43fe-ae57-308775a600c2 None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:27.451762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc46e5-b5f0-43fe-ae57-308775a600c2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:27.454660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 212/428] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:27 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:27.466876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc46e5-b5f0-43fe-ae57-308775a600c2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:27.510044 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:27.603373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0839c4-bdd8-41fa-a65c-7b3b03461263 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:27.616076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc46e5-b5f0-43fe-ae57-308775a600c2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:27.672266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82b7a8-c7d5-487c-8b25-7d4e1d61911c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:27.672860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82b7a8-c7d5-487c-8b25-7d4e1d61911c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:27.673450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82b7a8-c7d5-487c-8b25-7d4e1d61911c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:27.674116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82b7a8-c7d5-487c-8b25-7d4e1d61911c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:27.675186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82b7a8-c7d5-487c-8b25-7d4e1d61911c None admin] RBAC: Adding query filter params (group.id=f924c24d45a94a91be4b94e606ed767e, role.id=f4461b279e86422b83e29521e7ca29d0, scope.project.id=7139ad672c734cf799e4c39623384e9c) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:27.675658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82b7a8-c7d5-487c-8b25-7d4e1d61911c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:27.678112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82b7a8-c7d5-487c-8b25-7d4e1d61911c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:27.687398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 213/429] 10.222.0.93 () {56 vars in 1376 bytes} [Fri Jun 3 15:12:27 2022] GET /identity/v3/role_assignments?group.id=f924c24d45a94a91be4b94e606ed767e&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=f4461b279e86422b83e29521e7ca29d0 => generated 611 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:27.688882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208aaa6-7f87-42f8-91b2-b913a33544f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:27.690323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208aaa6-7f87-42f8-91b2-b913a33544f4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:27.690955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208aaa6-7f87-42f8-91b2-b913a33544f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:27.691410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208aaa6-7f87-42f8-91b2-b913a33544f4 None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:27.693254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208aaa6-7f87-42f8-91b2-b913a33544f4 None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:27.695129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208aaa6-7f87-42f8-91b2-b913a33544f4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:27.698353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 217/430] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:27 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 497 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:27.708189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82b7a8-c7d5-487c-8b25-7d4e1d61911c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:27.718598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ced5f-3ebd-482e-841c-34855477e3e0 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:27.719180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ced5f-3ebd-482e-841c-34855477e3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:27.719695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ced5f-3ebd-482e-841c-34855477e3e0 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:27.722940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 218/431] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:12:27.728130 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121747c6-cb29-4f8f-9bbc-1817a4ecfcde None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:27.728845 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121747c6-cb29-4f8f-9bbc-1817a4ecfcde None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:27.729315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121747c6-cb29-4f8f-9bbc-1817a4ecfcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:27.735389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:27.751203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-121747c6-cb29-4f8f-9bbc-1817a4ecfcde None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:27.753270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-121747c6-cb29-4f8f-9bbc-1817a4ecfcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:27.852548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 219/432] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:27.935115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:27.955483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82b7a8-c7d5-487c-8b25-7d4e1d61911c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:27.955759 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121747c6-cb29-4f8f-9bbc-1817a4ecfcde None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:28.018954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32596434-2756-4a59-9dcb-745a19cfaec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:28.020398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32596434-2756-4a59-9dcb-745a19cfaec3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:28.021040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32596434-2756-4a59-9dcb-745a19cfaec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:28.021692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32596434-2756-4a59-9dcb-745a19cfaec3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:28.023071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32596434-2756-4a59-9dcb-745a19cfaec3 None admin] RBAC: Adding query filter params (group.id=f924c24d45a94a91be4b94e606ed767e, role.id=1249af5cf5054410a11b2d88c03fcd52, scope.project.id=7139ad672c734cf799e4c39623384e9c) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:28.023743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32596434-2756-4a59-9dcb-745a19cfaec3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:28.025898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32596434-2756-4a59-9dcb-745a19cfaec3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:28.036147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 214/433] 10.222.0.93 () {56 vars in 1376 bytes} [Fri Jun 3 15:12:27 2022] GET /identity/v3/role_assignments?group.id=f924c24d45a94a91be4b94e606ed767e&scope.project.id=7139ad672c734cf799e4c39623384e9c&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 611 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:28.178231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121747c6-cb29-4f8f-9bbc-1817a4ecfcde None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:28.227040 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:28.233726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5373e24-00b5-4fe0-aefb-0aec9689faf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:28.234863 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373e24-00b5-4fe0-aefb-0aec9689faf4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:28.235200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373e24-00b5-4fe0-aefb-0aec9689faf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:28.235600 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5373e24-00b5-4fe0-aefb-0aec9689faf4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:28.256170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5373e24-00b5-4fe0-aefb-0aec9689faf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-c5373e24-00b5-4fe0-aefb-0aec9689faf4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:28.271247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:28.272220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 03 15:12:28.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:28.278925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 220/434] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:28.283923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32596434-2756-4a59-9dcb-745a19cfaec3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:28.427103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32596434-2756-4a59-9dcb-745a19cfaec3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:28.481761 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a562f37-f6f8-4eaf-8109-9123b007ec0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:28.482376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a562f37-f6f8-4eaf-8109-9123b007ec0e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:28.482719 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a562f37-f6f8-4eaf-8109-9123b007ec0e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:28.483010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a562f37-f6f8-4eaf-8109-9123b007ec0e None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:28.483784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a562f37-f6f8-4eaf-8109-9123b007ec0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:28.484095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a562f37-f6f8-4eaf-8109-9123b007ec0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:28.485518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a562f37-f6f8-4eaf-8109-9123b007ec0e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:28.497967 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 215/435] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:12:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:28.504135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5373e24-00b5-4fe0-aefb-0aec9689faf4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:28.589478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80805c-7217-405a-a82b-ed0448c456e9 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:28.589798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80805c-7217-405a-a82b-ed0448c456e9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:28.590086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80805c-7217-405a-a82b-ed0448c456e9 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:28.591559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 216/436] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:12:28.598564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008414f-b36d-4f2c-bee4-17b7c370dbd5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:28.598564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008414f-b36d-4f2c-bee4-17b7c370dbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:28.598970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008414f-b36d-4f2c-bee4-17b7c370dbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:28.631859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f008414f-b36d-4f2c-bee4-17b7c370dbd5 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:28.632426 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5373e24-00b5-4fe0-aefb-0aec9689faf4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:28.634144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f008414f-b36d-4f2c-bee4-17b7c370dbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:28.656194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:28.687335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f3439-903d-4efb-8e35-30dfefb6f681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:28.688457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f3439-903d-4efb-8e35-30dfefb6f681 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:28.688974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f3439-903d-4efb-8e35-30dfefb6f681 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:28.689490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f3439-903d-4efb-8e35-30dfefb6f681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:28.758807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 217/437] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:28.791572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008414f-b36d-4f2c-bee4-17b7c370dbd5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:28.799622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:28.825624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f3439-903d-4efb-8e35-30dfefb6f681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:28.828620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f3439-903d-4efb-8e35-30dfefb6f681 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:28.960505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008414f-b36d-4f2c-bee4-17b7c370dbd5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:28.976284 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 221/438] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:28.977832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228aaf0-0aed-4316-b796-45e2db32542d None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:28.978126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228aaf0-0aed-4316-b796-45e2db32542d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:28.978385 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228aaf0-0aed-4316-b796-45e2db32542d None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:28.982193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 222/439] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:28.983213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228aaf0-0aed-4316-b796-45e2db32542d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:29.008475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9828ce-a71f-4f61-b72d-8d3ac9ba674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:29.009491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9828ce-a71f-4f61-b72d-8d3ac9ba674f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:29.009669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9828ce-a71f-4f61-b72d-8d3ac9ba674f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:29.009930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9828ce-a71f-4f61-b72d-8d3ac9ba674f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:29.025225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9828ce-a71f-4f61-b72d-8d3ac9ba674f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-3a9828ce-a71f-4f61-b72d-8d3ac9ba674f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:29.035812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:29.037305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:29.038564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 218/440] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:12:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:29.039709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ccf0e-f830-49f6-9eff-8b48d8293331 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:29.040089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ccf0e-f830-49f6-9eff-8b48d8293331 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:29.040298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ccf0e-f830-49f6-9eff-8b48d8293331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:29.047367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:29.063308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-834ccf0e-f830-49f6-9eff-8b48d8293331 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:29.064533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-834ccf0e-f830-49f6-9eff-8b48d8293331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:29.102468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:29.114586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a228aaf0-0aed-4316-b796-45e2db32542d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:29.142887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 219/441] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:29.143759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ccf0e-f830-49f6-9eff-8b48d8293331 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:29.169781 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe966a5-c741-4381-af33-1570ef507df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:29.170697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe966a5-c741-4381-af33-1570ef507df9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:29.170982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe966a5-c741-4381-af33-1570ef507df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:29.171259 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe966a5-c741-4381-af33-1570ef507df9 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:29.172753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe966a5-c741-4381-af33-1570ef507df9 None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:29.174810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe966a5-c741-4381-af33-1570ef507df9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:29.183546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 223/442] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:28 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:29.183546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe966a5-c741-4381-af33-1570ef507df9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:29.267005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ccf0e-f830-49f6-9eff-8b48d8293331 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:29.311829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe966a5-c741-4381-af33-1570ef507df9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:29.320282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509690d-fd2e-430b-9964-711425ef6029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:29.321326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509690d-fd2e-430b-9964-711425ef6029 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:29.321792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509690d-fd2e-430b-9964-711425ef6029 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:29.322252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509690d-fd2e-430b-9964-711425ef6029 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:29.323596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509690d-fd2e-430b-9964-711425ef6029 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:29.324138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509690d-fd2e-430b-9964-711425ef6029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:29.326105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509690d-fd2e-430b-9964-711425ef6029 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:29.328667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:29.342620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 220/443] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:29.343986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509690d-fd2e-430b-9964-711425ef6029 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:29.374071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712fda7-59b6-48c5-ae7e-7a35006ac2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:29.374949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712fda7-59b6-48c5-ae7e-7a35006ac2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:29.375228 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712fda7-59b6-48c5-ae7e-7a35006ac2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:29.375529 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712fda7-59b6-48c5-ae7e-7a35006ac2e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:29.389697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712fda7-59b6-48c5-ae7e-7a35006ac2e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-a712fda7-59b6-48c5-ae7e-7a35006ac2e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:29.398024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:29.400140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 224/444] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:12:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:29.401410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712fda7-59b6-48c5-ae7e-7a35006ac2e2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:29.443184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:29.491215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509690d-fd2e-430b-9964-711425ef6029 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:29.542325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712fda7-59b6-48c5-ae7e-7a35006ac2e2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:29.547566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e1a96-e40e-4b82-93c0-559ce3a5fcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:29.548427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e1a96-e40e-4b82-93c0-559ce3a5fcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:29.548709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e1a96-e40e-4b82-93c0-559ce3a5fcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:29.548983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e1a96-e40e-4b82-93c0-559ce3a5fcc5 None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:29.550180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e1a96-e40e-4b82-93c0-559ce3a5fcc5 None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:29.551727 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e1a96-e40e-4b82-93c0-559ce3a5fcc5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:29.553355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 221/445] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:29 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:29.554124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e1a96-e40e-4b82-93c0-559ce3a5fcc5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:29.605049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd7205-e327-4085-8f31-cc5c727fdf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:29.605709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd7205-e327-4085-8f31-cc5c727fdf5b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:29.605990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd7205-e327-4085-8f31-cc5c727fdf5b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:29.606384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd7205-e327-4085-8f31-cc5c727fdf5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:29.649671 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:29.705583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd7205-e327-4085-8f31-cc5c727fdf5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:29.707213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd7205-e327-4085-8f31-cc5c727fdf5b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:29.709963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e1a96-e40e-4b82-93c0-559ce3a5fcc5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:29.729732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:29.756782 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df3c99-1381-4486-b012-fc014ffc22d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:29.757740 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df3c99-1381-4486-b012-fc014ffc22d2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:29.758172 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df3c99-1381-4486-b012-fc014ffc22d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:29.758674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df3c99-1381-4486-b012-fc014ffc22d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:29.759928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df3c99-1381-4486-b012-fc014ffc22d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:29.760424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df3c99-1381-4486-b012-fc014ffc22d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:29.762868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df3c99-1381-4486-b012-fc014ffc22d2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:29.780813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 222/446] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:29.780813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df3c99-1381-4486-b012-fc014ffc22d2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:29.839181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 225/447] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:29.844313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd7205-e327-4085-8f31-cc5c727fdf5b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:29.909378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df3c99-1381-4486-b012-fc014ffc22d2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:29.936019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:29.947584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9a412-83f4-4a72-8594-a4674f532b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:29.948428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9a412-83f4-4a72-8594-a4674f532b9e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:29.948726 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9a412-83f4-4a72-8594-a4674f532b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:29.949105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9a412-83f4-4a72-8594-a4674f532b9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:29.949949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9a412-83f4-4a72-8594-a4674f532b9e None admin] RBAC: Adding query filter params (group.id=f924c24d45a94a91be4b94e606ed767e, role.id=1249af5cf5054410a11b2d88c03fcd52, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:29.950255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9a412-83f4-4a72-8594-a4674f532b9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:29.951776 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e9a412-83f4-4a72-8594-a4674f532b9e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:29.956703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd7205-e327-4085-8f31-cc5c727fdf5b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:29.958263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 223/448] 10.222.0.93 () {56 vars in 1376 bytes} [Fri Jun 3 15:12:29 2022] GET /identity/v3/role_assignments?group.id=f924c24d45a94a91be4b94e606ed767e&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=1249af5cf5054410a11b2d88c03fcd52 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:29.959048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9a412-83f4-4a72-8594-a4674f532b9e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:29.987408 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:30.014467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cc3af-e275-4d6d-b484-1097e9edc821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:30.015171 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cc3af-e275-4d6d-b484-1097e9edc821 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:30.015452 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cc3af-e275-4d6d-b484-1097e9edc821 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:30.015856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30cc3af-e275-4d6d-b484-1097e9edc821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:30.103382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9a412-83f4-4a72-8594-a4674f532b9e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:30.118100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cc3af-e275-4d6d-b484-1097e9edc821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:30.121092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30cc3af-e275-4d6d-b484-1097e9edc821 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:30.144097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b2d97-7017-4316-9867-ebff47a25815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:30.144978 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2d97-7017-4316-9867-ebff47a25815 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:30.145257 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2d97-7017-4316-9867-ebff47a25815 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:30.145531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b2d97-7017-4316-9867-ebff47a25815 None admin] PATH_INFO: `/v3/projects/4b7e9c80374f4695bdf655157e8cb4d3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:30.147167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b2d97-7017-4316-9867-ebff47a25815 None admin] RBAC: Authorizing `identity:get_project(project_id=4b7e9c80374f4695bdf655157e8cb4d3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:30.148984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b2d97-7017-4316-9867-ebff47a25815 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:30.150772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 224/449] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:29 2022] GET /identity/v3/projects/4b7e9c80374f4695bdf655157e8cb4d3 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:30.156925 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b2d97-7017-4316-9867-ebff47a25815 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:30.249294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:30.277796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 226/450] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:30.283031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cc3af-e275-4d6d-b484-1097e9edc821 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:30.310249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:30.318028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b2d97-7017-4316-9867-ebff47a25815 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:30.374853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b09eb-cf90-4b07-85f2-39ef3ff27241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:30.375632 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b09eb-cf90-4b07-85f2-39ef3ff27241 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:30.375920 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b09eb-cf90-4b07-85f2-39ef3ff27241 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:30.376199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b09eb-cf90-4b07-85f2-39ef3ff27241 None admin] PATH_INFO: `/v3/groups/663850abe4df4cf287639abea09b0ef4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:30.378135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b09eb-cf90-4b07-85f2-39ef3ff27241 None admin] RBAC: Authorizing `identity:get_group(group_id=663850abe4df4cf287639abea09b0ef4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:30.379492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b09eb-cf90-4b07-85f2-39ef3ff27241 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:30.380595 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 225/451] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:30 2022] GET /identity/v3/groups/663850abe4df4cf287639abea09b0ef4 => generated 229 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:30.386716 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b09eb-cf90-4b07-85f2-39ef3ff27241 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:30.432824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30cc3af-e275-4d6d-b484-1097e9edc821 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:30.471748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad8787-da05-4d53-8070-2dd35ecd61c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:30.472608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8787-da05-4d53-8070-2dd35ecd61c5 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:30.472893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8787-da05-4d53-8070-2dd35ecd61c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:30.473169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad8787-da05-4d53-8070-2dd35ecd61c5 None admin] PATH_INFO: `/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:30.474423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad8787-da05-4d53-8070-2dd35ecd61c5 None admin] RBAC: Authorizing `identity:get_project(project_id=6590c6b28b6c4f22af7a8c90f8ce9cd9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:30.475902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad8787-da05-4d53-8070-2dd35ecd61c5 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:30.477642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 227/452] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:12:30 2022] GET /identity/v3/projects/6590c6b28b6c4f22af7a8c90f8ce9cd9 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:30.483252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad8787-da05-4d53-8070-2dd35ecd61c5 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:30.504478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b09eb-cf90-4b07-85f2-39ef3ff27241 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:30.531533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:30.542033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06d950-f52b-4298-8a3b-0ab65ddebb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:30.542724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06d950-f52b-4298-8a3b-0ab65ddebb3b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:30.543011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06d950-f52b-4298-8a3b-0ab65ddebb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:30.543293 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06d950-f52b-4298-8a3b-0ab65ddebb3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:30.544168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06d950-f52b-4298-8a3b-0ab65ddebb3b None admin] RBAC: Adding query filter params (group.id=663850abe4df4cf287639abea09b0ef4, role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=4b7e9c80374f4695bdf655157e8cb4d3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:30.544482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06d950-f52b-4298-8a3b-0ab65ddebb3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:30.545980 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06d950-f52b-4298-8a3b-0ab65ddebb3b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:30.552258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 226/453] 10.222.0.93 () {56 vars in 1376 bytes} [Fri Jun 3 15:12:30 2022] GET /identity/v3/role_assignments?group.id=663850abe4df4cf287639abea09b0ef4&scope.project.id=4b7e9c80374f4695bdf655157e8cb4d3&role.id=d67707782358417ba9f25b2793ca2890 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:30.618251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:30.646574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad8787-da05-4d53-8070-2dd35ecd61c5 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:30.703528 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f6322-bfa2-4359-820c-656a53d542eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:30.704396 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f6322-bfa2-4359-820c-656a53d542eb None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:30.704677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f6322-bfa2-4359-820c-656a53d542eb None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:30.704950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f6322-bfa2-4359-820c-656a53d542eb None admin] PATH_INFO: `/v3/groups/f924c24d45a94a91be4b94e606ed767e` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:30.706174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f6322-bfa2-4359-820c-656a53d542eb None admin] RBAC: Authorizing `identity:get_group(group_id=f924c24d45a94a91be4b94e606ed767e)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:30.707699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f6322-bfa2-4359-820c-656a53d542eb None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:30.709366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 228/454] 10.222.0.93 () {56 vars in 1180 bytes} [Fri Jun 3 15:12:30 2022] GET /identity/v3/groups/f924c24d45a94a91be4b94e606ed767e => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:30.716081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06d950-f52b-4298-8a3b-0ab65ddebb3b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:30.825816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06d950-f52b-4298-8a3b-0ab65ddebb3b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:30.858498 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e353f7-d76c-43cb-93b2-13d9d585ee2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:30.859159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e353f7-d76c-43cb-93b2-13d9d585ee2d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:30.859448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e353f7-d76c-43cb-93b2-13d9d585ee2d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:30.859911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e353f7-d76c-43cb-93b2-13d9d585ee2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:30.861569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e353f7-d76c-43cb-93b2-13d9d585ee2d None admin] RBAC: Adding query filter params (group.id=f924c24d45a94a91be4b94e606ed767e, role.id=f4461b279e86422b83e29521e7ca29d0, scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:30.861569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e353f7-d76c-43cb-93b2-13d9d585ee2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:30.862363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e353f7-d76c-43cb-93b2-13d9d585ee2d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:30.868219 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 227/455] 10.222.0.93 () {56 vars in 1376 bytes} [Fri Jun 3 15:12:30 2022] GET /identity/v3/role_assignments?group.id=f924c24d45a94a91be4b94e606ed767e&scope.project.id=6590c6b28b6c4f22af7a8c90f8ce9cd9&role.id=f4461b279e86422b83e29521e7ca29d0 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:34.148721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea58731-3f0d-44ef-bf3e-d7fbc3e7e2a5 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:34.149242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea58731-3f0d-44ef-bf3e-d7fbc3e7e2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:34.149630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea58731-3f0d-44ef-bf3e-d7fbc3e7e2a5 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:34.153412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 229/456] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:34.161896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6f95-07ba-4063-905b-e9f67f63ab70 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:34.163322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6f95-07ba-4063-905b-e9f67f63ab70 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:34.163936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd6f95-07ba-4063-905b-e9f67f63ab70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:34.193146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-cccd6f95-07ba-4063-905b-e9f67f63ab70 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:34.195072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cccd6f95-07ba-4063-905b-e9f67f63ab70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:34.283078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:34.306975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 228/457] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:34.343988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea58731-3f0d-44ef-bf3e-d7fbc3e7e2a5 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:34.520638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea58731-3f0d-44ef-bf3e-d7fbc3e7e2a5 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:34.572119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4a853-8280-46a4-8679-8f98b073d83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:34.573404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a853-8280-46a4-8679-8f98b073d83d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:34.573859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a853-8280-46a4-8679-8f98b073d83d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:34.574338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4a853-8280-46a4-8679-8f98b073d83d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:34.598562 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4a853-8280-46a4-8679-8f98b073d83d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:34.602222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4a853-8280-46a4-8679-8f98b073d83d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-9bc4a853-8280-46a4-8679-8f98b073d83d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:34.629131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:34.630043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:34.630910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:34.630910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:34.630910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:34.630910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:34.632521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 230/458] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:34.634875 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3884db-877b-4bb9-95df-d5940c56af09 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:34.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3884db-877b-4bb9-95df-d5940c56af09 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:34.635890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3884db-877b-4bb9-95df-d5940c56af09 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:34.638607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4a853-8280-46a4-8679-8f98b073d83d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:34.639793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 229/459] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:34.648287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb52b1-0a9e-4d53-9d16-ee2c4c4112f0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:34.648775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb52b1-0a9e-4d53-9d16-ee2c4c4112f0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:34.649276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb52b1-0a9e-4d53-9d16-ee2c4c4112f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:34.652847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:34.694833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-aabb52b1-0a9e-4d53-9d16-ee2c4c4112f0 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:34.697316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aabb52b1-0a9e-4d53-9d16-ee2c4c4112f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:34.814969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4a853-8280-46a4-8679-8f98b073d83d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:34.824765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 230/460] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:12:34.862375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e8b70-3083-47a4-b883-edbca1c301ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:34.863087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e8b70-3083-47a4-b883-edbca1c301ba None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:34.863367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e8b70-3083-47a4-b883-edbca1c301ba None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:34.863642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e8b70-3083-47a4-b883-edbca1c301ba None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:34.864380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e8b70-3083-47a4-b883-edbca1c301ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:34.864683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e8b70-3083-47a4-b883-edbca1c301ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:34.866077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e8b70-3083-47a4-b883-edbca1c301ba None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:34.880593 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb52b1-0a9e-4d53-9d16-ee2c4c4112f0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:34.886232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 231/461] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:34.893219 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e8b70-3083-47a4-b883-edbca1c301ba None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:35.016507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:35.046814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb52b1-0a9e-4d53-9d16-ee2c4c4112f0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:35.065746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:35.067468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e8b70-3083-47a4-b883-edbca1c301ba None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:35.093641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0b1bb-fcf6-4a77-9364-a56ea9cfab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:35.094443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0b1bb-fcf6-4a77-9364-a56ea9cfab6b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:35.094747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0b1bb-fcf6-4a77-9364-a56ea9cfab6b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:35.095022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0b1bb-fcf6-4a77-9364-a56ea9cfab6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:35.107607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0b1bb-fcf6-4a77-9364-a56ea9cfab6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:35.109260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc0b1bb-fcf6-4a77-9364-a56ea9cfab6b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:35.116063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a391285-d56a-4a64-bf78-5f9f57a3ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:35.117045 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391285-d56a-4a64-bf78-5f9f57a3ef12 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:35.117614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391285-d56a-4a64-bf78-5f9f57a3ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:35.117881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391285-d56a-4a64-bf78-5f9f57a3ef12 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:35.118638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a391285-d56a-4a64-bf78-5f9f57a3ef12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:35.120685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a391285-d56a-4a64-bf78-5f9f57a3ef12 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ccc0b1bb-fcf6-4a77-9364-a56ea9cfab6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:35.134666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:35.136071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:35.136921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:35.136921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:35.136921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:35.136921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:35.136921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 231/462] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:35.142454 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb085d5-59d8-4421-9ab4-47e2958f614d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:35.142759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb085d5-59d8-4421-9ab4-47e2958f614d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:35.143032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb085d5-59d8-4421-9ab4-47e2958f614d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:35.144494 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 232/463] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:35.155047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 232/464] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:12:34 2022] POST /identity/v3/users => generated 248 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:12:35.155047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a391285-d56a-4a64-bf78-5f9f57a3ef12 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:35.155692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd6f34-6b4f-4387-8046-f35a2343d951 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:35.155692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd6f34-6b4f-4387-8046-f35a2343d951 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:35.155692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facd6f34-6b4f-4387-8046-f35a2343d951 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:35.155692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 233/465] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:35.155692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fcb70-614f-469e-9c17-b7a5c4505c87 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:35.155692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fcb70-614f-469e-9c17-b7a5c4505c87 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:35.156346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047fcb70-614f-469e-9c17-b7a5c4505c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:35.206273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-047fcb70-614f-469e-9c17-b7a5c4505c87 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:35.208372 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-047fcb70-614f-469e-9c17-b7a5c4505c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:35.326401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a391285-d56a-4a64-bf78-5f9f57a3ef12 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:35.329227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 234/466] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:35.330876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b4fad-3154-4ed9-94f9-abcfe55c1608 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:35.331167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b4fad-3154-4ed9-94f9-abcfe55c1608 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:35.331438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b4fad-3154-4ed9-94f9-abcfe55c1608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:35.352863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c76b4fad-3154-4ed9-94f9-abcfe55c1608 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:35.354852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c76b4fad-3154-4ed9-94f9-abcfe55c1608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:35.364532 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:35.389332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07197af-770f-4f3c-a5fe-4afd4bc2eabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:35.390417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07197af-770f-4f3c-a5fe-4afd4bc2eabc None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:35.391012 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07197af-770f-4f3c-a5fe-4afd4bc2eabc None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:35.391539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07197af-770f-4f3c-a5fe-4afd4bc2eabc None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:35.392608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07197af-770f-4f3c-a5fe-4afd4bc2eabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:35.393139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07197af-770f-4f3c-a5fe-4afd4bc2eabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:35.394862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07197af-770f-4f3c-a5fe-4afd4bc2eabc None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:35.409602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 233/467] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:35.411487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07197af-770f-4f3c-a5fe-4afd4bc2eabc None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:35.470273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 235/468] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:35.472688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b4fad-3154-4ed9-94f9-abcfe55c1608 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:35.512729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:35.592994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07197af-770f-4f3c-a5fe-4afd4bc2eabc None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:35.646133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b4fad-3154-4ed9-94f9-abcfe55c1608 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:35.661300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dff2e-966c-4f1c-ae2e-b2b3949747ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:35.662732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dff2e-966c-4f1c-ae2e-b2b3949747ae None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:35.663135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dff2e-966c-4f1c-ae2e-b2b3949747ae None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:35.663431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6dff2e-966c-4f1c-ae2e-b2b3949747ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:35.681918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dff2e-966c-4f1c-ae2e-b2b3949747ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:35.684590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6dff2e-966c-4f1c-ae2e-b2b3949747ae None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:35.687438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed299c-4c4e-443e-bc0b-3bde3011456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:35.688152 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed299c-4c4e-443e-bc0b-3bde3011456b None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:35.688435 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed299c-4c4e-443e-bc0b-3bde3011456b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:35.688710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed299c-4c4e-443e-bc0b-3bde3011456b None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:35.689446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed299c-4c4e-443e-bc0b-3bde3011456b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:35.691228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed299c-4c4e-443e-bc0b-3bde3011456b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ce6dff2e-966c-4f1c-ae2e-b2b3949747ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:35.712005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:35.712948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:35.713812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:35.713812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:35.713812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:35.713812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:35.713812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 234/469] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:35.714831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dff2e-966c-4f1c-ae2e-b2b3949747ae None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:35.727371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 236/470] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:12:35 2022] POST /identity/v3/users => generated 246 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:12:35.729307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed299c-4c4e-443e-bc0b-3bde3011456b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:35.812483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:35.954451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6dff2e-966c-4f1c-ae2e-b2b3949747ae None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:35.986389 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:36.029139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600afbb-07d8-4794-8003-d478acbd5f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:36.029660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed299c-4c4e-443e-bc0b-3bde3011456b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:36.029952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600afbb-07d8-4794-8003-d478acbd5f50 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:36.030234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600afbb-07d8-4794-8003-d478acbd5f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:36.030506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600afbb-07d8-4794-8003-d478acbd5f50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:36.045288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600afbb-07d8-4794-8003-d478acbd5f50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:36.049445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f600afbb-07d8-4794-8003-d478acbd5f50 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-f600afbb-07d8-4794-8003-d478acbd5f50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:36.072174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:36.073097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:36.073964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:36.073964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:36.073964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:36.073964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:36.073964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 235/471] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:36.080984 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600afbb-07d8-4794-8003-d478acbd5f50 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:36.086134 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f2c71-9f48-44f1-bcc6-aaf15d17d9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:36.088006 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f2c71-9f48-44f1-bcc6-aaf15d17d9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:36.088506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f2c71-9f48-44f1-bcc6-aaf15d17d9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:36.088984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f2c71-9f48-44f1-bcc6-aaf15d17d9b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:36.089968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f2c71-9f48-44f1-bcc6-aaf15d17d9b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:36.091014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f2c71-9f48-44f1-bcc6-aaf15d17d9b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:36.092649 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f2c71-9f48-44f1-bcc6-aaf15d17d9b5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:36.115577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 237/472] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:36.117689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef3be-8b8f-49df-9fae-ee7abd1fd26d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:36.118497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef3be-8b8f-49df-9fae-ee7abd1fd26d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:36.119003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef3be-8b8f-49df-9fae-ee7abd1fd26d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:36.121005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 238/473] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:36.122002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aef3be-8b8f-49df-9fae-ee7abd1fd26d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:36.251127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:36.256721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aef3be-8b8f-49df-9fae-ee7abd1fd26d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:36.267083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600afbb-07d8-4794-8003-d478acbd5f50 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:36.295682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426958f2-955b-4736-aee8-5c11aa5c2ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:36.296361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426958f2-955b-4736-aee8-5c11aa5c2ef7 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:36.296697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426958f2-955b-4736-aee8-5c11aa5c2ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:36.296978 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426958f2-955b-4736-aee8-5c11aa5c2ef7 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:36.297691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426958f2-955b-4736-aee8-5c11aa5c2ef7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:36.299280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426958f2-955b-4736-aee8-5c11aa5c2ef7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:36.314146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c0015-daae-425b-aa8c-29755f1bd9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:36.315000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c0015-daae-425b-aa8c-29755f1bd9cb None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:36.315317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c0015-daae-425b-aa8c-29755f1bd9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:36.315721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:36.317104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c0015-daae-425b-aa8c-29755f1bd9cb None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:36.319082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c0015-daae-425b-aa8c-29755f1bd9cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:36.319691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c0015-daae-425b-aa8c-29755f1bd9cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:36.321263 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c0015-daae-425b-aa8c-29755f1bd9cb None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:36.326991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 239/474] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:12:36 2022] POST /identity/v3/users => generated 249 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:12:36.328469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf18e80-26e3-47e8-8ef3-1205abb22196 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:36.328851 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf18e80-26e3-47e8-8ef3-1205abb22196 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:36.329133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf18e80-26e3-47e8-8ef3-1205abb22196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:36.354765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 236/475] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:36.357702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c0015-daae-425b-aa8c-29755f1bd9cb None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:36.520390 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c0015-daae-425b-aa8c-29755f1bd9cb None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:36.568218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-adf18e80-26e3-47e8-8ef3-1205abb22196 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:36.570304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-adf18e80-26e3-47e8-8ef3-1205abb22196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:36.604089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b93a7-1657-4a34-b737-c1450ec8ddfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:36.604769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b93a7-1657-4a34-b737-c1450ec8ddfc None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:36.605047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b93a7-1657-4a34-b737-c1450ec8ddfc None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:36.605320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b93a7-1657-4a34-b737-c1450ec8ddfc None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:36.606085 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b93a7-1657-4a34-b737-c1450ec8ddfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:36.607560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974b93a7-1657-4a34-b737-c1450ec8ddfc None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:36.639967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 237/476] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:12:36 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:12:36.702675 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 240/477] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:36.724912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b93a7-1657-4a34-b737-c1450ec8ddfc None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:36.737220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:36.859144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b93a7-1657-4a34-b737-c1450ec8ddfc None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:36.897589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658e1ff-a296-4854-8581-9ce2857ee280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:36.898276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658e1ff-a296-4854-8581-9ce2857ee280 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:36.898615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658e1ff-a296-4854-8581-9ce2857ee280 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:36.898854 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658e1ff-a296-4854-8581-9ce2857ee280 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:36.899634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4658e1ff-a296-4854-8581-9ce2857ee280 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:36.901056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4658e1ff-a296-4854-8581-9ce2857ee280 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:36.908371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.sql.core [None req-4658e1ff-a296-4854-8581-9ce2857ee280 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 03 15:12:36.908371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 03 15:12:36.908371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [parameters: {'id': 'b10177cd633d4f2a8c0df4c7cc551268', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 03 15:12:36.908371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80221) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-4658e1ff-a296-4854-8581-9ce2857ee280 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application conn.query(q) Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result.read() Jun 03 15:12:36.926332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return ref.to_dict() Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 15:12:36.927855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application next(self.gen) Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application yield resource Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application next(self.gen) Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.session.rollback() Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.force_reraise() Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise self.value Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application session.commit() Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 03 15:12:36.929406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._prepare_impl() Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.session.flush() Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._flush(objects) Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application compat.raise_( Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application flush_context.execute() Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rec.execute(self) Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 03 15:12:36.930917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 15:12:36.932440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application conn.query(q) Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result.read() Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 15:12:36.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [parameters: {'id': 'b10177cd633d4f2a8c0df4c7cc551268', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:36.943437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 03 15:12:36.946695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:36.951265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 238/478] 10.222.0.93 () {60 vars in 1131 bytes} [Fri Jun 3 15:12:36 2022] POST /identity/v3/roles => generated 144 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:36.951474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf18e80-26e3-47e8-8ef3-1205abb22196 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:37.052141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf18e80-26e3-47e8-8ef3-1205abb22196 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:37.082179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:37.090949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616571ee-23c4-499b-af71-5c81c195bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:37.091970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616571ee-23c4-499b-af71-5c81c195bb05 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:37.092256 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616571ee-23c4-499b-af71-5c81c195bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:37.092530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616571ee-23c4-499b-af71-5c81c195bb05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:37.103800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616571ee-23c4-499b-af71-5c81c195bb05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-616571ee-23c4-499b-af71-5c81c195bb05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:37.112505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:37.113404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 03 15:12:37.114195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:37.118413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 241/479] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:12:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:37.120082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658e1ff-a296-4854-8581-9ce2857ee280 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:37.200302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:37.236831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658e1ff-a296-4854-8581-9ce2857ee280 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:37.278430 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d54eb-a178-4858-9a00-bc41fd99b605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:37.279221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d54eb-a178-4858-9a00-bc41fd99b605 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:37.279457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d54eb-a178-4858-9a00-bc41fd99b605 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:37.279739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d54eb-a178-4858-9a00-bc41fd99b605 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:37.280505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d54eb-a178-4858-9a00-bc41fd99b605 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:37.280813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d54eb-a178-4858-9a00-bc41fd99b605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:37.282199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d54eb-a178-4858-9a00-bc41fd99b605 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:37.295052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 239/480] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:38.159751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689d8f9-bd9a-4992-8fbb-2db95ef1605b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:38.160304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689d8f9-bd9a-4992-8fbb-2db95ef1605b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:38.160304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689d8f9-bd9a-4992-8fbb-2db95ef1605b None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:38.161957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 242/481] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:38.167491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ba840-0f98-44f5-9236-b402a91feb6d None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:38.167783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ba840-0f98-44f5-9236-b402a91feb6d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:38.168067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ba840-0f98-44f5-9236-b402a91feb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:38.191634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-7f6ba840-0f98-44f5-9236-b402a91feb6d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:38.192848 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7f6ba840-0f98-44f5-9236-b402a91feb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:38.292455 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 240/482] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:38.325801 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689d8f9-bd9a-4992-8fbb-2db95ef1605b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:38.467334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689d8f9-bd9a-4992-8fbb-2db95ef1605b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:38.516688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538d7aeb-0f47-41b1-8c5f-3e21ca48b550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:38.517492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d7aeb-0f47-41b1-8c5f-3e21ca48b550 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:38.517773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d7aeb-0f47-41b1-8c5f-3e21ca48b550 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:38.518046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538d7aeb-0f47-41b1-8c5f-3e21ca48b550 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:38.534499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538d7aeb-0f47-41b1-8c5f-3e21ca48b550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-538d7aeb-0f47-41b1-8c5f-3e21ca48b550 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:38.543051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:38.544581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:38.545367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:38.550554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 243/483] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:38.552417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ba840-0f98-44f5-9236-b402a91feb6d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:38.634973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:38.704972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ba840-0f98-44f5-9236-b402a91feb6d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:38.721553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33339aa-20bf-4dcb-b209-0e2130589c68 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:38.721889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33339aa-20bf-4dcb-b209-0e2130589c68 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:38.722297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33339aa-20bf-4dcb-b209-0e2130589c68 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:38.728891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 244/484] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:12:38.734664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c03577-899a-4ad6-baf5-8e54a8c182b1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:38.735172 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c03577-899a-4ad6-baf5-8e54a8c182b1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:38.735619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c03577-899a-4ad6-baf5-8e54a8c182b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:38.742669 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6576f7f-293f-4923-b358-bf36b01be9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:38.743611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6576f7f-293f-4923-b358-bf36b01be9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:38.744056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6576f7f-293f-4923-b358-bf36b01be9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:38.744477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6576f7f-293f-4923-b358-bf36b01be9d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:38.745389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6576f7f-293f-4923-b358-bf36b01be9d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:38.745841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6576f7f-293f-4923-b358-bf36b01be9d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:38.747471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6576f7f-293f-4923-b358-bf36b01be9d4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:38.757756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 241/485] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:38.765505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6576f7f-293f-4923-b358-bf36b01be9d4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:38.776609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f6c03577-899a-4ad6-baf5-8e54a8c182b1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:38.779166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f6c03577-899a-4ad6-baf5-8e54a8c182b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:38.879284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:38.915160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 245/486] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:12:38.945630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6576f7f-293f-4923-b358-bf36b01be9d4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:38.956144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c03577-899a-4ad6-baf5-8e54a8c182b1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:38.987401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:39.010836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19571d-5e6d-4e5d-b8d9-0220d7518a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:39.012491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19571d-5e6d-4e5d-b8d9-0220d7518a63 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:39.012923 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19571d-5e6d-4e5d-b8d9-0220d7518a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:39.013348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19571d-5e6d-4e5d-b8d9-0220d7518a63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:39.031465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19571d-5e6d-4e5d-b8d9-0220d7518a63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:39.033201 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19571d-5e6d-4e5d-b8d9-0220d7518a63 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-6a19571d-5e6d-4e5d-b8d9-0220d7518a63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:39.064794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:39.065890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:39.066972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:39.066972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:39.066972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:39.066972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:39.078755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 242/487] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:39.083157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19571d-5e6d-4e5d-b8d9-0220d7518a63 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:39.135765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c03577-899a-4ad6-baf5-8e54a8c182b1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:39.178187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d26b17-d536-4f51-8532-b67b7748db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:39.180039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26b17-d536-4f51-8532-b67b7748db91 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:39.180465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26b17-d536-4f51-8532-b67b7748db91 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:39.180878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d26b17-d536-4f51-8532-b67b7748db91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:39.193912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d26b17-d536-4f51-8532-b67b7748db91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-66d26b17-d536-4f51-8532-b67b7748db91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:39.203142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:39.204280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:39.205277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:39.207885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 246/488] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:39.214064 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d26b17-d536-4f51-8532-b67b7748db91 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:39.299028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:39.336356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19571d-5e6d-4e5d-b8d9-0220d7518a63 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:39.349276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d26b17-d536-4f51-8532-b67b7748db91 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:39.373867 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b808ec-4a34-4556-96a8-d2628bc03acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:39.374552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b808ec-4a34-4556-96a8-d2628bc03acf None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:39.374952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b808ec-4a34-4556-96a8-d2628bc03acf None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:39.375119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b808ec-4a34-4556-96a8-d2628bc03acf None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:39.375837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b808ec-4a34-4556-96a8-d2628bc03acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:39.376140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b808ec-4a34-4556-96a8-d2628bc03acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:39.377533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b808ec-4a34-4556-96a8-d2628bc03acf None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:39.411084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd17fa-d58d-4eab-a209-b3bb2cb1a3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:39.411766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd17fa-d58d-4eab-a209-b3bb2cb1a3db None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:39.412046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd17fa-d58d-4eab-a209-b3bb2cb1a3db None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:39.412312 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd17fa-d58d-4eab-a209-b3bb2cb1a3db None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:39.413060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd17fa-d58d-4eab-a209-b3bb2cb1a3db None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:39.413362 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd17fa-d58d-4eab-a209-b3bb2cb1a3db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:39.415063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befd17fa-d58d-4eab-a209-b3bb2cb1a3db None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:39.417010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 243/489] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:39.423218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b808ec-4a34-4556-96a8-d2628bc03acf None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:39.444130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 247/490] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:39.451900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd17fa-d58d-4eab-a209-b3bb2cb1a3db None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:39.562195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:39.616227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd17fa-d58d-4eab-a209-b3bb2cb1a3db None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:39.623167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:39.696336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d46fd-222b-4c2d-a031-00e75b241c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:39.697511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d46fd-222b-4c2d-a031-00e75b241c0c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:39.697790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d46fd-222b-4c2d-a031-00e75b241c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:39.698062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d46fd-222b-4c2d-a031-00e75b241c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:39.733309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d46fd-222b-4c2d-a031-00e75b241c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:39.739078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d46fd-222b-4c2d-a031-00e75b241c0c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-113d46fd-222b-4c2d-a031-00e75b241c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:39.798947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:39.800484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:39.802246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:39.802246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:39.802246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:39.802246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:39.802246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 248/491] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:39.805488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b808ec-4a34-4556-96a8-d2628bc03acf None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:39.806827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d46fd-222b-4c2d-a031-00e75b241c0c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:39.944011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6c745f-fd53-44dc-9986-cb820040ca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:39.944769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c745f-fd53-44dc-9986-cb820040ca8d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:39.945073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c745f-fd53-44dc-9986-cb820040ca8d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:39.945475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6c745f-fd53-44dc-9986-cb820040ca8d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:39.957523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c745f-fd53-44dc-9986-cb820040ca8d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:39.960383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6c745f-fd53-44dc-9986-cb820040ca8d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:39.969315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d46fd-222b-4c2d-a031-00e75b241c0c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ea6c745f-fd53-44dc-9986-cb820040ca8d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:39.990828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:39.991815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:39.992707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:39.992707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:39.992707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:39.992707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:39.992707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:39.992707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:39.992707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:39.992707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:39.992707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 03 15:12:39.992707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:39.993220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 244/492] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 570 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:39.998586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1ccc-2336-4370-82e0-a3a6df344bf9 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:39.998586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1ccc-2336-4370-82e0-a3a6df344bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:39.998586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1ccc-2336-4370-82e0-a3a6df344bf9 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:39.998586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 245/493] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:39.999990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1ccc-2336-4370-82e0-a3a6df344bf9 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:40.035891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ccffb-a90a-4f89-81af-b0f99dd9b74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:40.037402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ccffb-a90a-4f89-81af-b0f99dd9b74d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:40.038031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ccffb-a90a-4f89-81af-b0f99dd9b74d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:40.038759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ccffb-a90a-4f89-81af-b0f99dd9b74d None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:40.039939 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ccffb-a90a-4f89-81af-b0f99dd9b74d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:40.040808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ccffb-a90a-4f89-81af-b0f99dd9b74d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:40.042238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ccffb-a90a-4f89-81af-b0f99dd9b74d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:40.057112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 249/494] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:40.058339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b196d6-c34e-4982-96d4-4210105c505d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:40.058669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b196d6-c34e-4982-96d4-4210105c505d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:40.058982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b196d6-c34e-4982-96d4-4210105c505d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:40.078446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-15b196d6-c34e-4982-96d4-4210105c505d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:40.079722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-15b196d6-c34e-4982-96d4-4210105c505d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:40.094915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:40.159082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:40.175315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1ccc-2336-4370-82e0-a3a6df344bf9 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:40.180468 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 250/495] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:40.182769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94edf5f-abe3-4fca-b3bd-cde8401412dd None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:40.183312 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94edf5f-abe3-4fca-b3bd-cde8401412dd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:40.183812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94edf5f-abe3-4fca-b3bd-cde8401412dd None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:40.189678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 251/496] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:40.190749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94edf5f-abe3-4fca-b3bd-cde8401412dd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:40.238546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe0ab7-6b95-4e05-b2fd-8118ccbfbfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:40.239149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe0ab7-6b95-4e05-b2fd-8118ccbfbfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:40.239431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe0ab7-6b95-4e05-b2fd-8118ccbfbfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:40.239707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe0ab7-6b95-4e05-b2fd-8118ccbfbfd2 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:40.240559 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe0ab7-6b95-4e05-b2fd-8118ccbfbfd2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:40.240874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe0ab7-6b95-4e05-b2fd-8118ccbfbfd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:40.242354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe0ab7-6b95-4e05-b2fd-8118ccbfbfd2 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:40.297333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 246/497] 10.222.0.93 () {56 vars in 1137 bytes} [Fri Jun 3 15:12:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:40.297333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedf795-b9e0-4f10-84a9-faf9fd07e0b3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:40.297333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedf795-b9e0-4f10-84a9-faf9fd07e0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:40.297667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedf795-b9e0-4f10-84a9-faf9fd07e0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:40.320129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-bdedf795-b9e0-4f10-84a9-faf9fd07e0b3 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:40.321361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-bdedf795-b9e0-4f10-84a9-faf9fd07e0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:40.341261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94edf5f-abe3-4fca-b3bd-cde8401412dd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:40.390855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed85469-4733-4b1a-bbc6-6822ba32b26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:40.391737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed85469-4733-4b1a-bbc6-6822ba32b26e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:40.392015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed85469-4733-4b1a-bbc6-6822ba32b26e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:40.392292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed85469-4733-4b1a-bbc6-6822ba32b26e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:40.401871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed85469-4733-4b1a-bbc6-6822ba32b26e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:40.404180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed85469-4733-4b1a-bbc6-6822ba32b26e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:40.412890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 247/498] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:40.414409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedf795-b9e0-4f10-84a9-faf9fd07e0b3 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-1ed85469-4733-4b1a-bbc6-6822ba32b26e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:40.424165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:40.425121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:40.426007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:40.426007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:40.426007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:40.426007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:40.426007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:40.426007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:40.426007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:40.426007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:40.426007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:12:40.426007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:40.426007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 252/499] 10.222.0.93 () {56 vars in 1093 bytes} [Fri Jun 3 15:12:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:40.426605 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed85469-4733-4b1a-bbc6-6822ba32b26e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:40.436830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:40.519304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:40.537854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed85469-4733-4b1a-bbc6-6822ba32b26e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:40.543320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedf795-b9e0-4f10-84a9-faf9fd07e0b3 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:40.579027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf79801-4196-4ca7-9ed7-1665193a699c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:40.579678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf79801-4196-4ca7-9ed7-1665193a699c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:40.579958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf79801-4196-4ca7-9ed7-1665193a699c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:40.580229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf79801-4196-4ca7-9ed7-1665193a699c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:40.580830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cd8ad-7614-4397-ab64-761a0899b769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:40.581708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cd8ad-7614-4397-ab64-761a0899b769 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:40.582031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cd8ad-7614-4397-ab64-761a0899b769 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:40.582303 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cd8ad-7614-4397-ab64-761a0899b769 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:40.595916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156cd8ad-7614-4397-ab64-761a0899b769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-156cd8ad-7614-4397-ab64-761a0899b769 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:40.606056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:40.607704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 248/500] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:40.608719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5787c-f0aa-4562-a093-ffdb4c09d987 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:40.610493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5787c-f0aa-4562-a093-ffdb4c09d987 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:40.610493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d5787c-f0aa-4562-a093-ffdb4c09d987 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:40.610822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 249/501] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:40.611281 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5787c-f0aa-4562-a093-ffdb4c09d987 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:40.673915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf79801-4196-4ca7-9ed7-1665193a699c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:40.676867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf79801-4196-4ca7-9ed7-1665193a699c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:40.679760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:40.747494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d5787c-f0aa-4562-a093-ffdb4c09d987 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:40.793361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d441ad3-74a5-4944-a5eb-619b3c7725b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:40.794678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d441ad3-74a5-4944-a5eb-619b3c7725b2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:40.795131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d441ad3-74a5-4944-a5eb-619b3c7725b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:40.795512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d441ad3-74a5-4944-a5eb-619b3c7725b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:40.807553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d441ad3-74a5-4944-a5eb-619b3c7725b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-4d441ad3-74a5-4944-a5eb-619b3c7725b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:40.815733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:40.816878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 250/502] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:40.817739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d441ad3-74a5-4944-a5eb-619b3c7725b2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:40.821198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:40.826703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 253/503] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:40.828849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca566a4-57cc-4931-95d3-7f08f318e93b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:40.829345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca566a4-57cc-4931-95d3-7f08f318e93b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:40.829839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca566a4-57cc-4931-95d3-7f08f318e93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:40.849855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0ca566a4-57cc-4931-95d3-7f08f318e93b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:40.851465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0ca566a4-57cc-4931-95d3-7f08f318e93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:40.954558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 254/504] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:40.954558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca566a4-57cc-4931-95d3-7f08f318e93b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:40.954830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d441ad3-74a5-4944-a5eb-619b3c7725b2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:40.980321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:41.000504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823468d7-78c3-4247-92f9-ccc8f60a2963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:41.001165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823468d7-78c3-4247-92f9-ccc8f60a2963 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:41.001447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823468d7-78c3-4247-92f9-ccc8f60a2963 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:41.001723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823468d7-78c3-4247-92f9-ccc8f60a2963 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:41.002603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823468d7-78c3-4247-92f9-ccc8f60a2963 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:41.002916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823468d7-78c3-4247-92f9-ccc8f60a2963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:41.004354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823468d7-78c3-4247-92f9-ccc8f60a2963 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:41.027008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 251/505] 10.222.0.93 () {56 vars in 1133 bytes} [Fri Jun 3 15:12:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:41.027900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823468d7-78c3-4247-92f9-ccc8f60a2963 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:41.058670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca566a4-57cc-4931-95d3-7f08f318e93b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:41.110366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b871f8d-5b3e-41fa-bbaa-37f16a5055f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:41.111088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b871f8d-5b3e-41fa-bbaa-37f16a5055f0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:41.111368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b871f8d-5b3e-41fa-bbaa-37f16a5055f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:41.111639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b871f8d-5b3e-41fa-bbaa-37f16a5055f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:41.112397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b871f8d-5b3e-41fa-bbaa-37f16a5055f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:41.112700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b871f8d-5b3e-41fa-bbaa-37f16a5055f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:41.114367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b871f8d-5b3e-41fa-bbaa-37f16a5055f0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:41.124804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 255/506] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:41.125781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b871f8d-5b3e-41fa-bbaa-37f16a5055f0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:41.226006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:41.252459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:41.262797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b871f8d-5b3e-41fa-bbaa-37f16a5055f0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:41.279507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823468d7-78c3-4247-92f9-ccc8f60a2963 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:41.304288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee1de7-dec0-4a56-bd18-da086768006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:41.304965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee1de7-dec0-4a56-bd18-da086768006a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:41.305245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee1de7-dec0-4a56-bd18-da086768006a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:41.305523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ee1de7-dec0-4a56-bd18-da086768006a None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:41.306307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee1de7-dec0-4a56-bd18-da086768006a None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:41.306670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee1de7-dec0-4a56-bd18-da086768006a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:41.308066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ee1de7-dec0-4a56-bd18-da086768006a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:41.316756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf39e2-a122-4ad9-9833-5062f490a385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:41.317523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf39e2-a122-4ad9-9833-5062f490a385 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:41.317802 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf39e2-a122-4ad9-9833-5062f490a385 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:41.324782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf39e2-a122-4ad9-9833-5062f490a385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:41.325019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 256/507] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:41.325019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee1de7-dec0-4a56-bd18-da086768006a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:41.335533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf39e2-a122-4ad9-9833-5062f490a385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:41.337020 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf39e2-a122-4ad9-9833-5062f490a385 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-9eaf39e2-a122-4ad9-9833-5062f490a385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:41.356994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:41.357946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:41.358834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:41.358834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:41.358834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:41.358834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:41.358834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 252/508] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:41.359141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf39e2-a122-4ad9-9833-5062f490a385 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:41.445361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ee1de7-dec0-4a56-bd18-da086768006a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:41.470162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf39e2-a122-4ad9-9833-5062f490a385 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:41.484337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:41.485823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d17ee-c3b5-4b78-bffa-d474e849a60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:41.487030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d17ee-c3b5-4b78-bffa-d474e849a60d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:41.487449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d17ee-c3b5-4b78-bffa-d474e849a60d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:41.487885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d17ee-c3b5-4b78-bffa-d474e849a60d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:41.504278 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d17ee-c3b5-4b78-bffa-d474e849a60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:41.507034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d17ee-c3b5-4b78-bffa-d474e849a60d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:41.521850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fc132-cf36-4124-84a1-28a49206a3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:41.522513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fc132-cf36-4124-84a1-28a49206a3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:41.522826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fc132-cf36-4124-84a1-28a49206a3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:41.523103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fc132-cf36-4124-84a1-28a49206a3f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f25d17ee-c3b5-4b78-bffa-d474e849a60d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:41.531095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:41.532138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:41.533559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:41.533559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:41.533559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:41.533559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:41.533559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 257/509] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:41.533559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d17ee-c3b5-4b78-bffa-d474e849a60d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:41.552495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:41.606703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fc132-cf36-4124-84a1-28a49206a3f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:41.608124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492fc132-cf36-4124-84a1-28a49206a3f8 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:41.646589 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d17ee-c3b5-4b78-bffa-d474e849a60d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:41.687947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12301cd7-7ded-4d16-a1ce-5c9beae2cbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:41.688746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12301cd7-7ded-4d16-a1ce-5c9beae2cbcb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:41.689026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12301cd7-7ded-4d16-a1ce-5c9beae2cbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:41.689310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12301cd7-7ded-4d16-a1ce-5c9beae2cbcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:41.704111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12301cd7-7ded-4d16-a1ce-5c9beae2cbcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:41.705605 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12301cd7-7ded-4d16-a1ce-5c9beae2cbcb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:41.714991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 253/510] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:41.716591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fc132-cf36-4124-84a1-28a49206a3f8 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-12301cd7-7ded-4d16-a1ce-5c9beae2cbcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:41.727102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:41.728409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:41.729807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:41.729807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:41.729807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:41.729807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:41.729807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 258/511] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:41.729807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12301cd7-7ded-4d16-a1ce-5c9beae2cbcb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:41.736754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:41.797025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:41.844414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fc132-cf36-4124-84a1-28a49206a3f8 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:41.844624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12301cd7-7ded-4d16-a1ce-5c9beae2cbcb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:41.876993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d571a-93da-4ba4-b47e-fba9362817d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:41.877552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a47d6e-1ea2-4873-9ee9-0d3b4ea2cfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:41.877669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d571a-93da-4ba4-b47e-fba9362817d9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:41.877839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d571a-93da-4ba4-b47e-fba9362817d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:41.878115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d571a-93da-4ba4-b47e-fba9362817d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:41.878334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a47d6e-1ea2-4873-9ee9-0d3b4ea2cfca None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:41.878639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a47d6e-1ea2-4873-9ee9-0d3b4ea2cfca None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:41.878865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d571a-93da-4ba4-b47e-fba9362817d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:41.878953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a47d6e-1ea2-4873-9ee9-0d3b4ea2cfca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:41.879174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d571a-93da-4ba4-b47e-fba9362817d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:41.880664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d571a-93da-4ba4-b47e-fba9362817d9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:41.890865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a47d6e-1ea2-4873-9ee9-0d3b4ea2cfca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:41.892299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a47d6e-1ea2-4873-9ee9-0d3b4ea2cfca None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:41.892518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 259/512] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:41.896923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d571a-93da-4ba4-b47e-fba9362817d9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-10a47d6e-1ea2-4873-9ee9-0d3b4ea2cfca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:41.915274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:41.916300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:41.917180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:41.917180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:41.917180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:41.917180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:41.917180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 254/513] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:41.917725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a47d6e-1ea2-4873-9ee9-0d3b4ea2cfca None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:41.981350 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:41.998787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d571a-93da-4ba4-b47e-fba9362817d9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:42.020227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:42.020391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a47d6e-1ea2-4873-9ee9-0d3b4ea2cfca None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:42.034962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2595457-d17e-425a-8102-11a6ad9f8c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:42.035626 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2595457-d17e-425a-8102-11a6ad9f8c59 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:42.035910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2595457-d17e-425a-8102-11a6ad9f8c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:42.036207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2595457-d17e-425a-8102-11a6ad9f8c59 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:42.036919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2595457-d17e-425a-8102-11a6ad9f8c59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:42.037224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2595457-d17e-425a-8102-11a6ad9f8c59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:42.038626 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2595457-d17e-425a-8102-11a6ad9f8c59 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:42.049834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 260/514] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:42.050644 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2595457-d17e-425a-8102-11a6ad9f8c59 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:42.054789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d7a37-d3d1-4b1a-aef4-680e74b42f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:42.055579 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d7a37-d3d1-4b1a-aef4-680e74b42f02 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:42.055863 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d7a37-d3d1-4b1a-aef4-680e74b42f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:42.056140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d7a37-d3d1-4b1a-aef4-680e74b42f02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:42.067308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d7a37-d3d1-4b1a-aef4-680e74b42f02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:42.068794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d7a37-d3d1-4b1a-aef4-680e74b42f02 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ff9d7a37-d3d1-4b1a-aef4-680e74b42f02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:42.086481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:42.087375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:42.088230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:42.088230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:42.088230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:42.088230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:42.088230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 255/515] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:42.089131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d7a37-d3d1-4b1a-aef4-680e74b42f02 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:42.155339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2595457-d17e-425a-8102-11a6ad9f8c59 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:42.192447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c0173-4cd7-4cc6-a858-a92296da4be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:42.193117 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0173-4cd7-4cc6-a858-a92296da4be4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:42.193398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0173-4cd7-4cc6-a858-a92296da4be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:42.193681 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c0173-4cd7-4cc6-a858-a92296da4be4 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:42.194400 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c0173-4cd7-4cc6-a858-a92296da4be4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:42.194744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c0173-4cd7-4cc6-a858-a92296da4be4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:42.196133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8c0173-4cd7-4cc6-a858-a92296da4be4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:42.202271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d7a37-d3d1-4b1a-aef4-680e74b42f02 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:42.210620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 261/516] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:42.212106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c0173-4cd7-4cc6-a858-a92296da4be4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:42.224161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:42.243581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455744a7-0188-4660-8f5b-bb00ce74ace9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:42.244540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455744a7-0188-4660-8f5b-bb00ce74ace9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:42.244827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455744a7-0188-4660-8f5b-bb00ce74ace9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:42.245105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455744a7-0188-4660-8f5b-bb00ce74ace9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:42.253338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455744a7-0188-4660-8f5b-bb00ce74ace9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:42.254849 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455744a7-0188-4660-8f5b-bb00ce74ace9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:42.257876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-455744a7-0188-4660-8f5b-bb00ce74ace9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:42.271001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:42.272070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:42.273080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:42.273080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:42.273080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:42.273080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:42.273080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:42.273080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:42.273080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:42.273080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:42.273080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 256/517] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:42.273080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455744a7-0188-4660-8f5b-bb00ce74ace9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:42.323653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c0173-4cd7-4cc6-a858-a92296da4be4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:42.358492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7c29a-ba6f-479d-b5f6-dd56673437f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:42.359207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7c29a-ba6f-479d-b5f6-dd56673437f6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:42.359487 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7c29a-ba6f-479d-b5f6-dd56673437f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:42.359760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7c29a-ba6f-479d-b5f6-dd56673437f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:42.360471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7c29a-ba6f-479d-b5f6-dd56673437f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:42.360781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7c29a-ba6f-479d-b5f6-dd56673437f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:42.362158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7c29a-ba6f-479d-b5f6-dd56673437f6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:42.373911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 262/518] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:42.374786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7c29a-ba6f-479d-b5f6-dd56673437f6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:42.397048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455744a7-0188-4660-8f5b-bb00ce74ace9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:42.439391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1d69e-e104-4bdd-a114-b931e1ae2f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:42.439984 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1d69e-e104-4bdd-a114-b931e1ae2f81 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:42.440265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1d69e-e104-4bdd-a114-b931e1ae2f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:42.440541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1d69e-e104-4bdd-a114-b931e1ae2f81 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:42.441257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1d69e-e104-4bdd-a114-b931e1ae2f81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:42.442797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1d69e-e104-4bdd-a114-b931e1ae2f81 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:42.475995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 257/519] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:12:42 2022] POST /identity/v3/users => generated 247 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:12:42.475995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1d69e-e104-4bdd-a114-b931e1ae2f81 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:42.492495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:42.497830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7c29a-ba6f-479d-b5f6-dd56673437f6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:42.503790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:42.542800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532da50-8631-4cd4-a04e-695d26d57ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:42.544055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532da50-8631-4cd4-a04e-695d26d57ada None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:42.544343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532da50-8631-4cd4-a04e-695d26d57ada None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:42.544627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532da50-8631-4cd4-a04e-695d26d57ada None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:42.545371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532da50-8631-4cd4-a04e-695d26d57ada None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:42.545681 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532da50-8631-4cd4-a04e-695d26d57ada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:42.547172 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532da50-8631-4cd4-a04e-695d26d57ada None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:42.561324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 263/520] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:42.562081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532da50-8631-4cd4-a04e-695d26d57ada None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:42.592347 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1d69e-e104-4bdd-a114-b931e1ae2f81 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:42.635010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e451af0-1012-4d8a-8da8-98bd93e44db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:42.636144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e451af0-1012-4d8a-8da8-98bd93e44db9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:42.636620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e451af0-1012-4d8a-8da8-98bd93e44db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:42.637115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e451af0-1012-4d8a-8da8-98bd93e44db9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:42.648509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e451af0-1012-4d8a-8da8-98bd93e44db9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:42.650557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e451af0-1012-4d8a-8da8-98bd93e44db9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-4e451af0-1012-4d8a-8da8-98bd93e44db9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:42.667765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:42.669272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:42.670338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:42.670338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:42.670338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:42.670338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:42.670338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:42.670338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:42.670338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:42.670338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:42.670338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 15:12:42.670338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:42.670338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 258/521] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:42.670929 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e451af0-1012-4d8a-8da8-98bd93e44db9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:42.690957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532da50-8631-4cd4-a04e-695d26d57ada None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:42.742860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a974fa-b8bd-4cbb-8d1a-2ae6e114f05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:42.743537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a974fa-b8bd-4cbb-8d1a-2ae6e114f05a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:42.743818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a974fa-b8bd-4cbb-8d1a-2ae6e114f05a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:42.744093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a974fa-b8bd-4cbb-8d1a-2ae6e114f05a None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:42.744882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a974fa-b8bd-4cbb-8d1a-2ae6e114f05a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:42.745200 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a974fa-b8bd-4cbb-8d1a-2ae6e114f05a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:42.747280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a974fa-b8bd-4cbb-8d1a-2ae6e114f05a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:42.766268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 264/522] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:42.771165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a974fa-b8bd-4cbb-8d1a-2ae6e114f05a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:42.790447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:42.792376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:42.803343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e451af0-1012-4d8a-8da8-98bd93e44db9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:42.850253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53173a7c-abcb-45ef-bb2a-068c9e482ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:42.851204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53173a7c-abcb-45ef-bb2a-068c9e482ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:42.851397 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53173a7c-abcb-45ef-bb2a-068c9e482ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:42.851698 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53173a7c-abcb-45ef-bb2a-068c9e482ff2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:42.863985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53173a7c-abcb-45ef-bb2a-068c9e482ff2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:42.865650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53173a7c-abcb-45ef-bb2a-068c9e482ff2 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-53173a7c-abcb-45ef-bb2a-068c9e482ff2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:42.892944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:42.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:42.895420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:42.895420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:42.895420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:42.895420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:42.895420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:42.895420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:42.895420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:42.895420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:42.895420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 15:12:42.895420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:42.895420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 259/523] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:42.896202 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53173a7c-abcb-45ef-bb2a-068c9e482ff2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:42.903378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a974fa-b8bd-4cbb-8d1a-2ae6e114f05a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:42.947730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9be536-cbd9-4ff7-80dc-2c5312cb01d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:42.948280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9be536-cbd9-4ff7-80dc-2c5312cb01d6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:42.948633 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9be536-cbd9-4ff7-80dc-2c5312cb01d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:42.948913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9be536-cbd9-4ff7-80dc-2c5312cb01d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:42.962021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9be536-cbd9-4ff7-80dc-2c5312cb01d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:42.963617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9be536-cbd9-4ff7-80dc-2c5312cb01d6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ef9be536-cbd9-4ff7-80dc-2c5312cb01d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:42.983135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:42.984061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:42.984909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:42.984909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:42.984909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:42.984909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:42.984909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:42.984909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:42.984909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:42.984909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:42.984909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 265/524] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:42.985466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9be536-cbd9-4ff7-80dc-2c5312cb01d6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:43.020810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53173a7c-abcb-45ef-bb2a-068c9e482ff2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:43.060942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3651b04-9a5d-4d3d-b5d1-cd01a6558242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:43.061732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3651b04-9a5d-4d3d-b5d1-cd01a6558242 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:43.062017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3651b04-9a5d-4d3d-b5d1-cd01a6558242 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:43.062292 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3651b04-9a5d-4d3d-b5d1-cd01a6558242 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:43.063192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3651b04-9a5d-4d3d-b5d1-cd01a6558242 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:43.063501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3651b04-9a5d-4d3d-b5d1-cd01a6558242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:43.065063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3651b04-9a5d-4d3d-b5d1-cd01a6558242 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:43.092711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 260/525] 10.222.0.93 () {56 vars in 1139 bytes} [Fri Jun 3 15:12:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:43.092711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3651b04-9a5d-4d3d-b5d1-cd01a6558242 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:43.093087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:43.111077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:43.122446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9be536-cbd9-4ff7-80dc-2c5312cb01d6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:43.167781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafbfa3-5de3-4aef-a530-efe8c9e56476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:43.169021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafbfa3-5de3-4aef-a530-efe8c9e56476 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:43.169302 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafbfa3-5de3-4aef-a530-efe8c9e56476 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:43.169572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eafbfa3-5de3-4aef-a530-efe8c9e56476 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:43.170403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafbfa3-5de3-4aef-a530-efe8c9e56476 None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=13121a263c31433dbd4a5082b4bf3bb5) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:43.170756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafbfa3-5de3-4aef-a530-efe8c9e56476 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:43.172206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eafbfa3-5de3-4aef-a530-efe8c9e56476 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:43.178298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 266/526] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:12:42 2022] GET /identity/v3/role_assignments?user.id=13121a263c31433dbd4a5082b4bf3bb5&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:43.183699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafbfa3-5de3-4aef-a530-efe8c9e56476 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:43.231588 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3651b04-9a5d-4d3d-b5d1-cd01a6558242 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:43.286023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3536e6-ae60-451a-bd25-7169693a2bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:43.287220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3536e6-ae60-451a-bd25-7169693a2bba None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:43.288530 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3536e6-ae60-451a-bd25-7169693a2bba None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:43.289329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3536e6-ae60-451a-bd25-7169693a2bba None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:43.290453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3536e6-ae60-451a-bd25-7169693a2bba None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:43.291773 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3536e6-ae60-451a-bd25-7169693a2bba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:43.294310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3536e6-ae60-451a-bd25-7169693a2bba None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:43.326522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 261/527] 10.222.0.93 () {56 vars in 1137 bytes} [Fri Jun 3 15:12:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:43.327753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3536e6-ae60-451a-bd25-7169693a2bba None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:43.341169 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eafbfa3-5de3-4aef-a530-efe8c9e56476 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:43.397989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d9605-4e4c-41b6-93ce-593e6d8f0457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:43.398990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d9605-4e4c-41b6-93ce-593e6d8f0457 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:43.399473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d9605-4e4c-41b6-93ce-593e6d8f0457 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:43.400298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d9605-4e4c-41b6-93ce-593e6d8f0457 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:43.401598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d9605-4e4c-41b6-93ce-593e6d8f0457 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:43.402103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d9605-4e4c-41b6-93ce-593e6d8f0457 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:43.405357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d9605-4e4c-41b6-93ce-593e6d8f0457 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:43.422321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 267/528] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:43.423491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d9605-4e4c-41b6-93ce-593e6d8f0457 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:43.435769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:43.444553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:43.490832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3536e6-ae60-451a-bd25-7169693a2bba None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:43.542980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a3d0a-1c5b-4258-9f72-1808bc8948cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:43.544511 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a3d0a-1c5b-4258-9f72-1808bc8948cb None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:43.545472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a3d0a-1c5b-4258-9f72-1808bc8948cb None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:43.546003 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a3d0a-1c5b-4258-9f72-1808bc8948cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:43.580486 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d9605-4e4c-41b6-93ce-593e6d8f0457 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:43.619596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075ebd-ab14-4af2-8708-682281632325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:43.620282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075ebd-ab14-4af2-8708-682281632325 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:43.620613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075ebd-ab14-4af2-8708-682281632325 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:43.620894 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b075ebd-ab14-4af2-8708-682281632325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:43.649653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a3d0a-1c5b-4258-9f72-1808bc8948cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:43.651312 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a3d0a-1c5b-4258-9f72-1808bc8948cb None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:43.731413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b075ebd-ab14-4af2-8708-682281632325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:43.733526 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b075ebd-ab14-4af2-8708-682281632325 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:43.737012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:43.749273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:43.796042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 262/529] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:43.796952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a3d0a-1c5b-4258-9f72-1808bc8948cb None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:43.867278 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 268/530] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:43.868443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075ebd-ab14-4af2-8708-682281632325 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:43.912226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a3d0a-1c5b-4258-9f72-1808bc8948cb None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:43.950347 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e42737-0ef3-47e4-bb85-113ceabf592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:43.951310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e42737-0ef3-47e4-bb85-113ceabf592f None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:43.951594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e42737-0ef3-47e4-bb85-113ceabf592f None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:43.951870 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e42737-0ef3-47e4-bb85-113ceabf592f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:43.952877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e42737-0ef3-47e4-bb85-113ceabf592f None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=c1966524091242e7a6cd2758027a0d4f) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:43.953053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e42737-0ef3-47e4-bb85-113ceabf592f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:43.954451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e42737-0ef3-47e4-bb85-113ceabf592f None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:43.966560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 263/531] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:12:43 2022] GET /identity/v3/role_assignments?user.id=c1966524091242e7a6cd2758027a0d4f&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:43.966560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e42737-0ef3-47e4-bb85-113ceabf592f None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:43.998903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:44.019658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b075ebd-ab14-4af2-8708-682281632325 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:44.032239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:44.075078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e42737-0ef3-47e4-bb85-113ceabf592f None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:44.090933 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3b59d-d720-4d93-b39d-a5a9335a1e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:44.092140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3b59d-d720-4d93-b39d-a5a9335a1e62 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:44.092616 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3b59d-d720-4d93-b39d-a5a9335a1e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:44.093129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3b59d-d720-4d93-b39d-a5a9335a1e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:44.109098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3b59d-d720-4d93-b39d-a5a9335a1e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:44.110993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3b59d-d720-4d93-b39d-a5a9335a1e62 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:44.116521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899ece2-389d-483f-b53c-26635022b04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:44.116521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899ece2-389d-483f-b53c-26635022b04a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:44.116521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899ece2-389d-483f-b53c-26635022b04a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:44.116521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899ece2-389d-483f-b53c-26635022b04a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:44.134598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899ece2-389d-483f-b53c-26635022b04a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:44.134598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899ece2-389d-483f-b53c-26635022b04a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-cfd3b59d-d720-4d93-b39d-a5a9335a1e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:44.146063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:44.147079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:44.147915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:44.147915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:44.147915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:44.147915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-1899ece2-389d-483f-b53c-26635022b04a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:44.154743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:44.155654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:44.156475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:44.156475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:44.156475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:44.156475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:44.156475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 264/532] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:44.156749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 269/533] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:44.158664 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899ece2-389d-483f-b53c-26635022b04a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:44.167625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3b59d-d720-4d93-b39d-a5a9335a1e62 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:44.305776 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:44.325336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899ece2-389d-483f-b53c-26635022b04a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:44.353569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3b59d-d720-4d93-b39d-a5a9335a1e62 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:44.366563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37f584-cee2-4d6c-bf33-4c57590a1ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:44.366563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37f584-cee2-4d6c-bf33-4c57590a1ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:44.366563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37f584-cee2-4d6c-bf33-4c57590a1ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:44.366563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d37f584-cee2-4d6c-bf33-4c57590a1ab1 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:44.366971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37f584-cee2-4d6c-bf33-4c57590a1ab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:44.366971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37f584-cee2-4d6c-bf33-4c57590a1ab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:44.367307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d37f584-cee2-4d6c-bf33-4c57590a1ab1 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:44.382555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 265/534] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:44.385414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37f584-cee2-4d6c-bf33-4c57590a1ab1 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:44.421937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0d31c-aae9-4c4c-a843-b64c9ef94a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:44.422636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0d31c-aae9-4c4c-a843-b64c9ef94a1a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:44.422920 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0d31c-aae9-4c4c-a843-b64c9ef94a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:44.423196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0d31c-aae9-4c4c-a843-b64c9ef94a1a None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:44.423927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0d31c-aae9-4c4c-a843-b64c9ef94a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:44.424230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0d31c-aae9-4c4c-a843-b64c9ef94a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:44.425611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0d31c-aae9-4c4c-a843-b64c9ef94a1a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:44.443454 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 270/535] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:44.448734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0d31c-aae9-4c4c-a843-b64c9ef94a1a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:44.505647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:44.513248 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d37f584-cee2-4d6c-bf33-4c57590a1ab1 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:44.581771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a46a7-b2f1-4185-a116-0b9d932acdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:44.582947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a46a7-b2f1-4185-a116-0b9d932acdbe None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:44.583536 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a46a7-b2f1-4185-a116-0b9d932acdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:44.584024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a46a7-b2f1-4185-a116-0b9d932acdbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:44.599451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a46a7-b2f1-4185-a116-0b9d932acdbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:44.600935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a46a7-b2f1-4185-a116-0b9d932acdbe None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:44.613230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0d31c-aae9-4c4c-a843-b64c9ef94a1a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-f16a46a7-b2f1-4185-a116-0b9d932acdbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:44.639409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:44.640339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:44.641184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:44.641184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:44.641184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:44.641184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:44.641184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:44.641184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:44.641184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:44.641184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:44.650287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 266/536] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:44.659237 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a46a7-b2f1-4185-a116-0b9d932acdbe None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:44.665883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e91d0-7a96-4ca5-9364-60cc0a60ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:44.666590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:44.668988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e91d0-7a96-4ca5-9364-60cc0a60ab9b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:44.674305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e91d0-7a96-4ca5-9364-60cc0a60ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:44.674915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e91d0-7a96-4ca5-9364-60cc0a60ab9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:44.688933 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e91d0-7a96-4ca5-9364-60cc0a60ab9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:44.690399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e91d0-7a96-4ca5-9364-60cc0a60ab9b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ff8e91d0-7a96-4ca5-9364-60cc0a60ab9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:44.739783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:44.740692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:44.741534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:44.741534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:44.741534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:44.741534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:44.741534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:44.741534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:44.741534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:44.741534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:44.747217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 271/537] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:44.749259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e91d0-7a96-4ca5-9364-60cc0a60ab9b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:44.908114 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:44.909755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a46a7-b2f1-4185-a116-0b9d932acdbe None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:44.942904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e91d0-7a96-4ca5-9364-60cc0a60ab9b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:44.949544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46303f-a7e0-40e3-a08c-6fe55ae2a8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:44.950222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46303f-a7e0-40e3-a08c-6fe55ae2a8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:44.950507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46303f-a7e0-40e3-a08c-6fe55ae2a8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:44.950816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46303f-a7e0-40e3-a08c-6fe55ae2a8d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:44.951623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46303f-a7e0-40e3-a08c-6fe55ae2a8d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:44.952380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46303f-a7e0-40e3-a08c-6fe55ae2a8d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:44.953838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46303f-a7e0-40e3-a08c-6fe55ae2a8d6 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:44.982630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 267/538] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:44.984048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99e8e-9743-4ca5-b341-d1c296de30bd None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:44.984330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99e8e-9743-4ca5-b341-d1c296de30bd None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:44.984600 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e99e8e-9743-4ca5-b341-d1c296de30bd None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:44.985878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 268/539] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:44.986573 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e99e8e-9743-4ca5-b341-d1c296de30bd None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:44.989597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185bd6b-fd09-46ec-8947-55fe11c7ee4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:44.990267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185bd6b-fd09-46ec-8947-55fe11c7ee4d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:44.990557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185bd6b-fd09-46ec-8947-55fe11c7ee4d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:44.990842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185bd6b-fd09-46ec-8947-55fe11c7ee4d None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:44.991635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185bd6b-fd09-46ec-8947-55fe11c7ee4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:44.992001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185bd6b-fd09-46ec-8947-55fe11c7ee4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:44.993557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185bd6b-fd09-46ec-8947-55fe11c7ee4d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:45.009541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 272/540] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:45.015749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8e816-1683-4761-ac62-ee835fb2d3fa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:45.016033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8e816-1683-4761-ac62-ee835fb2d3fa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:45.016315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8e816-1683-4761-ac62-ee835fb2d3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:45.036185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-66c8e816-1683-4761-ac62-ee835fb2d3fa None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:45.037417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-66c8e816-1683-4761-ac62-ee835fb2d3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:45.074033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:45.123697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e99e8e-9743-4ca5-b341-d1c296de30bd None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:45.143929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 273/541] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:45.145118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8e816-1683-4761-ac62-ee835fb2d3fa None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:45.190639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57e375-d989-42f8-897b-68e882d577b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:45.191604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57e375-d989-42f8-897b-68e882d577b0 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:45.192086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57e375-d989-42f8-897b-68e882d577b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:45.192553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57e375-d989-42f8-897b-68e882d577b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:45.193577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57e375-d989-42f8-897b-68e882d577b0 None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=98aa51c4bc604b0ca8a6c69bc1455bbb) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:45.194108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57e375-d989-42f8-897b-68e882d577b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:45.195799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57e375-d989-42f8-897b-68e882d577b0 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:45.206684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 269/542] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:12:44 2022] GET /identity/v3/role_assignments?user.id=98aa51c4bc604b0ca8a6c69bc1455bbb&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:45.217429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57e375-d989-42f8-897b-68e882d577b0 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:45.224779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:45.273821 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8e816-1683-4761-ac62-ee835fb2d3fa None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:45.322949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a0c6b-cab4-4198-a5cf-2cbbff4e35a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:45.324026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a0c6b-cab4-4198-a5cf-2cbbff4e35a5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:45.324539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a0c6b-cab4-4198-a5cf-2cbbff4e35a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:45.325560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a0c6b-cab4-4198-a5cf-2cbbff4e35a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:45.329041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a0c6b-cab4-4198-a5cf-2cbbff4e35a5 None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=e3f887d5bbcd424884537b95affd96f8) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:45.329814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a0c6b-cab4-4198-a5cf-2cbbff4e35a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:45.331661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965a0c6b-cab4-4198-a5cf-2cbbff4e35a5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:45.349448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 274/543] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:12:45 2022] GET /identity/v3/role_assignments?user.id=e3f887d5bbcd424884537b95affd96f8&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:45.428123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57e375-d989-42f8-897b-68e882d577b0 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:45.473907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:45.489674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba0407-e52d-47f7-ae87-0befe8312858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:45.490484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba0407-e52d-47f7-ae87-0befe8312858 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:45.490735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba0407-e52d-47f7-ae87-0befe8312858 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:45.490994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba0407-e52d-47f7-ae87-0befe8312858 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:45.508679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba0407-e52d-47f7-ae87-0befe8312858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-38ba0407-e52d-47f7-ae87-0befe8312858 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:45.517159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:45.518594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:45.519404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 270/544] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:45.524680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a0c6b-cab4-4198-a5cf-2cbbff4e35a5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:45.710374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a0c6b-cab4-4198-a5cf-2cbbff4e35a5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:45.749556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:45.762288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d4540-87a6-4c19-b053-628f6e8c510a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:45.763082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d4540-87a6-4c19-b053-628f6e8c510a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:45.763367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d4540-87a6-4c19-b053-628f6e8c510a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:45.763683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d4540-87a6-4c19-b053-628f6e8c510a None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:45.764444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d4540-87a6-4c19-b053-628f6e8c510a None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:45.764748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d4540-87a6-4c19-b053-628f6e8c510a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:45.766423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d4540-87a6-4c19-b053-628f6e8c510a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:45.780955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 275/545] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:45.782368 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba0407-e52d-47f7-ae87-0befe8312858 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:45.923962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba0407-e52d-47f7-ae87-0befe8312858 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:45.972943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558d0b2-6a96-44c7-b64d-1edf59c64e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:45.974111 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558d0b2-6a96-44c7-b64d-1edf59c64e57 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:45.974629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558d0b2-6a96-44c7-b64d-1edf59c64e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:45.975342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558d0b2-6a96-44c7-b64d-1edf59c64e57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:45.978038 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e148e1-cf47-462b-80ac-a9ac761f09f6 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:45.978558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e148e1-cf47-462b-80ac-a9ac761f09f6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:45.979035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e148e1-cf47-462b-80ac-a9ac761f09f6 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:45.981767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 276/546] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:12:45.988251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558d0b2-6a96-44c7-b64d-1edf59c64e57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:45.990036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558d0b2-6a96-44c7-b64d-1edf59c64e57 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:45.991304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4539f6d-692e-415a-a7ce-c7a6fa837f2f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:45.991794 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4539f6d-692e-415a-a7ce-c7a6fa837f2f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:45.992274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4539f6d-692e-415a-a7ce-c7a6fa837f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-1558d0b2-6a96-44c7-b64d-1edf59c64e57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:46.010299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:46.011206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:46.012622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:46.012622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:46.012622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:46.012622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:46.020272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 271/547] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:46.021770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558d0b2-6a96-44c7-b64d-1edf59c64e57 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:46.040935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a4539f6d-692e-415a-a7ce-c7a6fa837f2f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:46.042873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a4539f6d-692e-415a-a7ce-c7a6fa837f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:46.125570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 277/548] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:46.141154 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:46.197485 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4539f6d-692e-415a-a7ce-c7a6fa837f2f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:46.218715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558d0b2-6a96-44c7-b64d-1edf59c64e57 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:46.263542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b0fb5-54ff-4158-936c-9d90d7fadeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:46.264966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b0fb5-54ff-4158-936c-9d90d7fadeb7 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:46.265518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b0fb5-54ff-4158-936c-9d90d7fadeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:46.266601 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b0fb5-54ff-4158-936c-9d90d7fadeb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:46.267563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b0fb5-54ff-4158-936c-9d90d7fadeb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:46.268367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b0fb5-54ff-4158-936c-9d90d7fadeb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:46.269996 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b0fb5-54ff-4158-936c-9d90d7fadeb7 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:46.287796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 272/549] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:46.291179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b0fb5-54ff-4158-936c-9d90d7fadeb7 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:46.409277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b0fb5-54ff-4158-936c-9d90d7fadeb7 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:46.411583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:46.453028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6939b7-de84-4da3-acd7-bcb3f18033d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:46.454211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6939b7-de84-4da3-acd7-bcb3f18033d4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:46.454747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6939b7-de84-4da3-acd7-bcb3f18033d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:46.455254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6939b7-de84-4da3-acd7-bcb3f18033d4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:46.463703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6939b7-de84-4da3-acd7-bcb3f18033d4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:46.465702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6939b7-de84-4da3-acd7-bcb3f18033d4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:46.503928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4539f6d-692e-415a-a7ce-c7a6fa837f2f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-3a6939b7-de84-4da3-acd7-bcb3f18033d4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:46.510299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:46.511677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:46.512585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:46.512585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:46.512585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:46.512585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:46.512585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:46.512585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:46.512585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:46.512585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:46.512585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:12:46.512585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:46.525600 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 273/550] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:12:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:46.536940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6939b7-de84-4da3-acd7-bcb3f18033d4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:46.559127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36426609-e2e2-4424-82d7-825a1f6339e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:46.560689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36426609-e2e2-4424-82d7-825a1f6339e3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:46.561188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36426609-e2e2-4424-82d7-825a1f6339e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:46.561692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36426609-e2e2-4424-82d7-825a1f6339e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:46.563025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:46.575308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36426609-e2e2-4424-82d7-825a1f6339e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-36426609-e2e2-4424-82d7-825a1f6339e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:46.584403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:46.585536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:46.586352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:46.589376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 278/551] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:46.602011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36426609-e2e2-4424-82d7-825a1f6339e3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:46.732762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36426609-e2e2-4424-82d7-825a1f6339e3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:46.775019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d068d-4480-4658-a049-cb2f61e0ce49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:46.776472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d068d-4480-4658-a049-cb2f61e0ce49 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:46.777177 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d068d-4480-4658-a049-cb2f61e0ce49 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:46.778320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d068d-4480-4658-a049-cb2f61e0ce49 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:46.780016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d068d-4480-4658-a049-cb2f61e0ce49 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:46.780877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d068d-4480-4658-a049-cb2f61e0ce49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:46.783660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d068d-4480-4658-a049-cb2f61e0ce49 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:46.797392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 279/552] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:46.807471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d068d-4480-4658-a049-cb2f61e0ce49 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:46.880177 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:46.938920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6939b7-de84-4da3-acd7-bcb3f18033d4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:46.983164 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d068d-4480-4658-a049-cb2f61e0ce49 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:46.987825 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc348a-2926-4bd1-bc3b-c0f283ac72c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:46.988754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc348a-2926-4bd1-bc3b-c0f283ac72c4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:46.989182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc348a-2926-4bd1-bc3b-c0f283ac72c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:46.989603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc348a-2926-4bd1-bc3b-c0f283ac72c4 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:46.990632 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc348a-2926-4bd1-bc3b-c0f283ac72c4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:46.991189 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc348a-2926-4bd1-bc3b-c0f283ac72c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:46.992807 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc348a-2926-4bd1-bc3b-c0f283ac72c4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:47.026270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 274/553] 10.222.0.93 () {56 vars in 1135 bytes} [Fri Jun 3 15:12:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 490 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:47.029670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961928d6-f409-4e94-bbfd-dde8d81f3e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:47.030003 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e0bf7-d63d-42d7-ac1b-f1ca9b369b5a None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:47.030385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961928d6-f409-4e94-bbfd-dde8d81f3e82 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:47.030718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961928d6-f409-4e94-bbfd-dde8d81f3e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:47.030997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961928d6-f409-4e94-bbfd-dde8d81f3e82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:47.031415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e0bf7-d63d-42d7-ac1b-f1ca9b369b5a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:47.032064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e0bf7-d63d-42d7-ac1b-f1ca9b369b5a None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:47.040283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 275/554] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:47.041716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e0bf7-d63d-42d7-ac1b-f1ca9b369b5a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:47.049265 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961928d6-f409-4e94-bbfd-dde8d81f3e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:47.051313 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961928d6-f409-4e94-bbfd-dde8d81f3e82 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-961928d6-f409-4e94-bbfd-dde8d81f3e82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:47.070501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:47.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:47.075120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:47.075120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:47.075120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:47.075120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:47.075120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 280/555] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:47.075120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dcbd5-4e45-4b93-b66f-a6d1b2c0d997 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:47.075120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dcbd5-4e45-4b93-b66f-a6d1b2c0d997 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:47.075120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04dcbd5-4e45-4b93-b66f-a6d1b2c0d997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:47.088000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:47.098674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d04dcbd5-4e45-4b93-b66f-a6d1b2c0d997 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:47.099872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d04dcbd5-4e45-4b93-b66f-a6d1b2c0d997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:47.196383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e0bf7-d63d-42d7-ac1b-f1ca9b369b5a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:47.210563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 281/556] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:47.210563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04dcbd5-4e45-4b93-b66f-a6d1b2c0d997 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:47.252569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:47.258173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930ed25-56a8-4a72-915d-a7ac02854837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:47.259220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930ed25-56a8-4a72-915d-a7ac02854837 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:47.259708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930ed25-56a8-4a72-915d-a7ac02854837 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:47.260180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e930ed25-56a8-4a72-915d-a7ac02854837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:47.361249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930ed25-56a8-4a72-915d-a7ac02854837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:47.364602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e930ed25-56a8-4a72-915d-a7ac02854837 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:47.388453 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04dcbd5-4e45-4b93-b66f-a6d1b2c0d997 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:47.390337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:47.450851 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d3b90-571f-4094-9c9c-326133a40193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:47.451996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d3b90-571f-4094-9c9c-326133a40193 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:47.452528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d3b90-571f-4094-9c9c-326133a40193 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:47.453015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d3b90-571f-4094-9c9c-326133a40193 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:47.453947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d3b90-571f-4094-9c9c-326133a40193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:47.454451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d3b90-571f-4094-9c9c-326133a40193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:47.457034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d3b90-571f-4094-9c9c-326133a40193 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:47.472749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 282/557] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:47.473998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d3b90-571f-4094-9c9c-326133a40193 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:47.503053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 276/558] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:47.503968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930ed25-56a8-4a72-915d-a7ac02854837 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:47.575417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:47.594769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d3b90-571f-4094-9c9c-326133a40193 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:47.628295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e930ed25-56a8-4a72-915d-a7ac02854837 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:47.635348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3df1dc-bd61-4847-ac82-4f0177a0e249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:47.636471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df1dc-bd61-4847-ac82-4f0177a0e249 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:47.637507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df1dc-bd61-4847-ac82-4f0177a0e249 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:47.638095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df1dc-bd61-4847-ac82-4f0177a0e249 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:47.650214 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3df1dc-bd61-4847-ac82-4f0177a0e249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:47.658714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2c3df1dc-bd61-4847-ac82-4f0177a0e249 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:47.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:47.666961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:47.668491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:47.673459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 283/559] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:47.674831 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3df1dc-bd61-4847-ac82-4f0177a0e249 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:47.693550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c50a18-c9bc-48c6-a700-17167bde3b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:47.694885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c50a18-c9bc-48c6-a700-17167bde3b71 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:47.695488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c50a18-c9bc-48c6-a700-17167bde3b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:47.696085 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c50a18-c9bc-48c6-a700-17167bde3b71 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:47.706073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c50a18-c9bc-48c6-a700-17167bde3b71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:47.707827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c50a18-c9bc-48c6-a700-17167bde3b71 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-07c50a18-c9bc-48c6-a700-17167bde3b71 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:47.724124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:47.725314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:47.726268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:47.726268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:47.726268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:47.726268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:47.726268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:47.726268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:47.726268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:47.726268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:47.726268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 03 15:12:47.726268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:47.726268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 277/560] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:47.726878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c50a18-c9bc-48c6-a700-17167bde3b71 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:47.806191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3df1dc-bd61-4847-ac82-4f0177a0e249 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:47.850655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814c888-cdc6-423f-a651-ddf8a858ac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:47.852163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814c888-cdc6-423f-a651-ddf8a858ac81 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:47.852987 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814c888-cdc6-423f-a651-ddf8a858ac81 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:47.853712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814c888-cdc6-423f-a651-ddf8a858ac81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:47.859338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c50a18-c9bc-48c6-a700-17167bde3b71 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:47.873521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814c888-cdc6-423f-a651-ddf8a858ac81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:47.875861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814c888-cdc6-423f-a651-ddf8a858ac81 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:47.887663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:47.901214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754aaf9-d2f9-4da8-8ef9-35259fff1941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:47.901953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754aaf9-d2f9-4da8-8ef9-35259fff1941 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:47.902485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754aaf9-d2f9-4da8-8ef9-35259fff1941 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:47.902964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754aaf9-d2f9-4da8-8ef9-35259fff1941 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9814c888-cdc6-423f-a651-ddf8a858ac81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:47.903559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:47.904960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:47.905969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:47.905969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:47.905969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:47.905969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:47.906232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754aaf9-d2f9-4da8-8ef9-35259fff1941 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:47.906232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754aaf9-d2f9-4da8-8ef9-35259fff1941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:47.906232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754aaf9-d2f9-4da8-8ef9-35259fff1941 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:47.908910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 284/561] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:47.910068 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814c888-cdc6-423f-a651-ddf8a858ac81 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:47.916412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 278/562] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:47.917310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754aaf9-d2f9-4da8-8ef9-35259fff1941 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:47.968681 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:48.037951 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754aaf9-d2f9-4da8-8ef9-35259fff1941 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:48.056599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814c888-cdc6-423f-a651-ddf8a858ac81 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:48.101483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea873aa-a9e7-4f97-9a6e-e51a73e515ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:48.102152 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a0dd2-2588-4b84-9e27-5d121a99faa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:48.102276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea873aa-a9e7-4f97-9a6e-e51a73e515ba None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:48.102657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea873aa-a9e7-4f97-9a6e-e51a73e515ba None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:48.103078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea873aa-a9e7-4f97-9a6e-e51a73e515ba None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:48.103739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0dd2-2588-4b84-9e27-5d121a99faa0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:48.103904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea873aa-a9e7-4f97-9a6e-e51a73e515ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:48.104166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea873aa-a9e7-4f97-9a6e-e51a73e515ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:48.104936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0dd2-2588-4b84-9e27-5d121a99faa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:48.106266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a0dd2-2588-4b84-9e27-5d121a99faa0 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:48.106426 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea873aa-a9e7-4f97-9a6e-e51a73e515ba None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:48.108088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a0dd2-2588-4b84-9e27-5d121a99faa0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:48.108701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a0dd2-2588-4b84-9e27-5d121a99faa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:48.111265 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a0dd2-2588-4b84-9e27-5d121a99faa0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:48.123035 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 279/563] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:48.124307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea873aa-a9e7-4f97-9a6e-e51a73e515ba None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:48.173375 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 285/564] 10.222.0.93 () {56 vars in 1137 bytes} [Fri Jun 3 15:12:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:48.175336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a0dd2-2588-4b84-9e27-5d121a99faa0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:48.252905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:48.268472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea873aa-a9e7-4f97-9a6e-e51a73e515ba None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:48.291299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:48.325578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a0dd2-2588-4b84-9e27-5d121a99faa0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:48.335848 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2684d76-5ea5-4393-8e75-543c32f082f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:48.337389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2684d76-5ea5-4393-8e75-543c32f082f4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:48.337885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2684d76-5ea5-4393-8e75-543c32f082f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:48.338598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2684d76-5ea5-4393-8e75-543c32f082f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:48.361015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2684d76-5ea5-4393-8e75-543c32f082f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:48.363902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2684d76-5ea5-4393-8e75-543c32f082f4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:48.366549 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ce29-1ea4-46be-bcd6-828c6285554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:48.367605 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ce29-1ea4-46be-bcd6-828c6285554b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:48.368033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ce29-1ea4-46be-bcd6-828c6285554b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:48.368534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65ce29-1ea4-46be-bcd6-828c6285554b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:48.377763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65ce29-1ea4-46be-bcd6-828c6285554b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:48.379658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65ce29-1ea4-46be-bcd6-828c6285554b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7d65ce29-1ea4-46be-bcd6-828c6285554b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:48.397710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:48.398889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:48.400018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:48.400018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:48.400018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:48.400018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:48.400018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:48.400018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:48.400018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:48.400018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-d2684d76-5ea5-4393-8e75-543c32f082f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:48.400550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:48.401549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:48.403270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:48.403270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:48.403270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:48.403270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:48.403270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 280/565] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:48.406564 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2684d76-5ea5-4393-8e75-543c32f082f4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:48.406707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 286/566] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:48.413297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ce29-1ea4-46be-bcd6-828c6285554b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:48.579094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65ce29-1ea4-46be-bcd6-828c6285554b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:48.579729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2684d76-5ea5-4393-8e75-543c32f082f4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:48.613744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:48.640816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0641ebc4-7f8c-4573-a3b6-711160ef6b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:48.641987 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641ebc4-7f8c-4573-a3b6-711160ef6b1b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:48.642413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641ebc4-7f8c-4573-a3b6-711160ef6b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:48.643050 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0641ebc4-7f8c-4573-a3b6-711160ef6b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:48.644070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f00bcf-4edc-438e-b6b3-681edbab1303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:48.645159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f00bcf-4edc-438e-b6b3-681edbab1303 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:48.645639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f00bcf-4edc-438e-b6b3-681edbab1303 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:48.646106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f00bcf-4edc-438e-b6b3-681edbab1303 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:48.647579 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f00bcf-4edc-438e-b6b3-681edbab1303 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:48.648110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f00bcf-4edc-438e-b6b3-681edbab1303 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:48.652811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f00bcf-4edc-438e-b6b3-681edbab1303 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:48.660003 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641ebc4-7f8c-4573-a3b6-711160ef6b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:48.662234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0641ebc4-7f8c-4573-a3b6-711160ef6b1b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:48.672989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 287/567] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:48.674322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f00bcf-4edc-438e-b6b3-681edbab1303 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-0641ebc4-7f8c-4573-a3b6-711160ef6b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:48.685425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:48.686514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:48.687449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:48.687449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:48.687449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:48.687449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:48.687449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 281/568] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:48.688268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4d03d-3875-4a31-881f-74967ac2c4a4 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:48.688556 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4d03d-3875-4a31-881f-74967ac2c4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:48.688831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4d03d-3875-4a31-881f-74967ac2c4a4 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:48.690250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 282/569] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:48.691177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e362d4-6f9b-46a6-a6f1-2a81ca09b3ce None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:48.691455 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e362d4-6f9b-46a6-a6f1-2a81ca09b3ce None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:48.691733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e362d4-6f9b-46a6-a6f1-2a81ca09b3ce None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:48.692925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 283/570] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:48.693611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e362d4-6f9b-46a6-a6f1-2a81ca09b3ce None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:48.697783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:48.814621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f00bcf-4edc-438e-b6b3-681edbab1303 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:48.856688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e362d4-6f9b-46a6-a6f1-2a81ca09b3ce None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:48.858752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72da820-34e5-4d93-a53d-5fbd0b93c983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:48.859440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72da820-34e5-4d93-a53d-5fbd0b93c983 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:48.859722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72da820-34e5-4d93-a53d-5fbd0b93c983 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:48.859999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72da820-34e5-4d93-a53d-5fbd0b93c983 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:48.861385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72da820-34e5-4d93-a53d-5fbd0b93c983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:48.861385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72da820-34e5-4d93-a53d-5fbd0b93c983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:48.862489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72da820-34e5-4d93-a53d-5fbd0b93c983 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:48.880062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 288/571] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:48.881407 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72da820-34e5-4d93-a53d-5fbd0b93c983 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:48.902904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489af815-2d40-46c8-bc21-c00e2f7c9b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:48.903640 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af815-2d40-46c8-bc21-c00e2f7c9b74 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:48.903942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af815-2d40-46c8-bc21-c00e2f7c9b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:48.904120 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af815-2d40-46c8-bc21-c00e2f7c9b74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:48.904973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489af815-2d40-46c8-bc21-c00e2f7c9b74 None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=bb4fb8f859dc4489b8c4fd8fdc09c3f6) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:48.905313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489af815-2d40-46c8-bc21-c00e2f7c9b74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:48.907001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489af815-2d40-46c8-bc21-c00e2f7c9b74 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:48.913905 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 284/572] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:12:48 2022] GET /identity/v3/role_assignments?user.id=bb4fb8f859dc4489b8c4fd8fdc09c3f6&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:48.915320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943d52b-a854-408a-b030-9cc5caf9ca91 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:48.915645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943d52b-a854-408a-b030-9cc5caf9ca91 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:48.915949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943d52b-a854-408a-b030-9cc5caf9ca91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:48.929339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:48.946104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-3943d52b-a854-408a-b030-9cc5caf9ca91 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:48.947557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3943d52b-a854-408a-b030-9cc5caf9ca91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:49.048102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72da820-34e5-4d93-a53d-5fbd0b93c983 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:49.073810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:49.082444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 285/573] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:49.084185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01e9a7-5286-4508-a419-4db649443545 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:49.084482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01e9a7-5286-4508-a419-4db649443545 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:49.084763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01e9a7-5286-4508-a419-4db649443545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:49.094591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c1edb-1d2f-4dfc-a106-7d7cfa29d33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:49.095952 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c1edb-1d2f-4dfc-a106-7d7cfa29d33c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:49.096761 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c1edb-1d2f-4dfc-a106-7d7cfa29d33c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:49.097309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32c1edb-1d2f-4dfc-a106-7d7cfa29d33c None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:49.098272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c1edb-1d2f-4dfc-a106-7d7cfa29d33c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:49.098833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c1edb-1d2f-4dfc-a106-7d7cfa29d33c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:49.100503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32c1edb-1d2f-4dfc-a106-7d7cfa29d33c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:49.107500 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-8f01e9a7-5286-4508-a419-4db649443545 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:49.108713 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8f01e9a7-5286-4508-a419-4db649443545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:49.123828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 289/574] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:49.123828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c1edb-1d2f-4dfc-a106-7d7cfa29d33c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:49.215819 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 286/575] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:49.217269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01e9a7-5286-4508-a419-4db649443545 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:49.279172 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:49.290112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32c1edb-1d2f-4dfc-a106-7d7cfa29d33c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:49.345804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4025c55-a419-461f-8911-fb1d744c46db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:49.346648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4025c55-a419-461f-8911-fb1d744c46db None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:49.347000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4025c55-a419-461f-8911-fb1d744c46db None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:49.347291 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4025c55-a419-461f-8911-fb1d744c46db None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:49.359978 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4025c55-a419-461f-8911-fb1d744c46db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:49.362226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4025c55-a419-461f-8911-fb1d744c46db None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b4025c55-a419-461f-8911-fb1d744c46db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:49.389778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:49.391832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:49.393648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:49.393648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:49.393648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:49.393648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:49.393648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:49.393648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:49.393648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:49.393648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:49.393648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:12:49.393648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:49.397506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 290/576] 10.222.0.93 () {56 vars in 1093 bytes} [Fri Jun 3 15:12:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:49.398771 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4025c55-a419-461f-8911-fb1d744c46db None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:49.418083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01e9a7-5286-4508-a419-4db649443545 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:49.457191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:49.473586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81340d7-c90e-46f2-87b2-937c1001195f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:49.474631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81340d7-c90e-46f2-87b2-937c1001195f None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:49.474808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81340d7-c90e-46f2-87b2-937c1001195f None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:49.475062 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81340d7-c90e-46f2-87b2-937c1001195f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:49.489822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81340d7-c90e-46f2-87b2-937c1001195f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-c81340d7-c90e-46f2-87b2-937c1001195f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:49.501739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:49.503727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:49.504838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:49.506957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 287/577] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:49.508056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81340d7-c90e-46f2-87b2-937c1001195f None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:49.548592 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4025c55-a419-461f-8911-fb1d744c46db None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:49.609821 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d09027-d491-4c1c-bd0a-d3e8f467fe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:49.611810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09027-d491-4c1c-bd0a-d3e8f467fe6c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:49.611810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09027-d491-4c1c-bd0a-d3e8f467fe6c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:49.611810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09027-d491-4c1c-bd0a-d3e8f467fe6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:49.620640 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d09027-d491-4c1c-bd0a-d3e8f467fe6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:49.622715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d09027-d491-4c1c-bd0a-d3e8f467fe6c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d0d09027-d491-4c1c-bd0a-d3e8f467fe6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:49.638641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:49.639717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:49.640993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:49.640993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:49.640993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:49.640993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:49.640993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:49.640993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:49.640993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:49.640993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:49.640993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 291/578] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:49.640993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d09027-d491-4c1c-bd0a-d3e8f467fe6c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:49.665491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:49.717822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81340d7-c90e-46f2-87b2-937c1001195f None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:49.781516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d96118-7aba-4404-9cfa-049d8efd74da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:49.783186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d96118-7aba-4404-9cfa-049d8efd74da None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:49.783839 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d96118-7aba-4404-9cfa-049d8efd74da None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:49.784483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d96118-7aba-4404-9cfa-049d8efd74da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:49.804379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d96118-7aba-4404-9cfa-049d8efd74da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:49.813122 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d09027-d491-4c1c-bd0a-d3e8f467fe6c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-01d96118-7aba-4404-9cfa-049d8efd74da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:49.818182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:49.819806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 288/579] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:49.826590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d96118-7aba-4404-9cfa-049d8efd74da None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:49.877058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3057d-6ffd-436b-bcf4-c0760082ceca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:49.878933 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3057d-6ffd-436b-bcf4-c0760082ceca None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:49.880308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3057d-6ffd-436b-bcf4-c0760082ceca None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:49.880928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3057d-6ffd-436b-bcf4-c0760082ceca None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:49.882011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3057d-6ffd-436b-bcf4-c0760082ceca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:49.882587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3057d-6ffd-436b-bcf4-c0760082ceca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:49.884657 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3057d-6ffd-436b-bcf4-c0760082ceca None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:49.909373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:49.930621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 292/580] 10.222.0.93 () {56 vars in 1133 bytes} [Fri Jun 3 15:12:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:49.930621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3057d-6ffd-436b-bcf4-c0760082ceca None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:49.989820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d96118-7aba-4404-9cfa-049d8efd74da None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:50.053769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf7dc1-6dc4-418b-80b2-fe02df5dadc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:50.054723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf7dc1-6dc4-418b-80b2-fe02df5dadc7 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:50.055194 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf7dc1-6dc4-418b-80b2-fe02df5dadc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:50.055613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf7dc1-6dc4-418b-80b2-fe02df5dadc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:50.056957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf7dc1-6dc4-418b-80b2-fe02df5dadc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:50.057480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf7dc1-6dc4-418b-80b2-fe02df5dadc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:50.059501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bf7dc1-6dc4-418b-80b2-fe02df5dadc7 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:50.071534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 289/581] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:50.073420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf7dc1-6dc4-418b-80b2-fe02df5dadc7 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:50.080482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:50.113955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3057d-6ffd-436b-bcf4-c0760082ceca None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:50.191960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330d6f5-f139-4223-ac43-36df0dd31a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:50.192884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330d6f5-f139-4223-ac43-36df0dd31a24 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:50.195451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330d6f5-f139-4223-ac43-36df0dd31a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:50.195451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b330d6f5-f139-4223-ac43-36df0dd31a24 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:50.195451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330d6f5-f139-4223-ac43-36df0dd31a24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:50.196531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330d6f5-f139-4223-ac43-36df0dd31a24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:50.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b330d6f5-f139-4223-ac43-36df0dd31a24 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:50.229350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 293/582] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:50.229350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330d6f5-f139-4223-ac43-36df0dd31a24 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:50.307794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf7dc1-6dc4-418b-80b2-fe02df5dadc7 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:50.328943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:50.369777 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fda634-62e4-4c89-b702-58749445c485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:50.371513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fda634-62e4-4c89-b702-58749445c485 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:50.372023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fda634-62e4-4c89-b702-58749445c485 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:50.372542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fda634-62e4-4c89-b702-58749445c485 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:50.374012 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fda634-62e4-4c89-b702-58749445c485 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:50.374582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fda634-62e4-4c89-b702-58749445c485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:50.377357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fda634-62e4-4c89-b702-58749445c485 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:50.393751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 290/583] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:50.394387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fda634-62e4-4c89-b702-58749445c485 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:50.410238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b330d6f5-f139-4223-ac43-36df0dd31a24 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:50.476690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49642fcd-501e-4621-9392-0a9135eb74bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:50.477993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642fcd-501e-4621-9392-0a9135eb74bb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:50.478473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642fcd-501e-4621-9392-0a9135eb74bb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:50.479084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49642fcd-501e-4621-9392-0a9135eb74bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:50.483942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:50.501296 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49642fcd-501e-4621-9392-0a9135eb74bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:50.503666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49642fcd-501e-4621-9392-0a9135eb74bb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-49642fcd-501e-4621-9392-0a9135eb74bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:50.529659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:50.531246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:50.532383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:50.532383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:50.532383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:50.532383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:50.534882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 294/584] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:50.536232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49642fcd-501e-4621-9392-0a9135eb74bb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:50.575772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fda634-62e4-4c89-b702-58749445c485 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:50.642773 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71896751-0fcb-407d-93b3-40e2dd5616b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:50.644078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71896751-0fcb-407d-93b3-40e2dd5616b3 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:50.644557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71896751-0fcb-407d-93b3-40e2dd5616b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:50.645036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71896751-0fcb-407d-93b3-40e2dd5616b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:50.663156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71896751-0fcb-407d-93b3-40e2dd5616b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:50.666192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71896751-0fcb-407d-93b3-40e2dd5616b3 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-71896751-0fcb-407d-93b3-40e2dd5616b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:50.703465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:50.704422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:50.705315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:50.705315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:50.705315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:50.705315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:50.706341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 291/585] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:50.707135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71896751-0fcb-407d-93b3-40e2dd5616b3 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:50.743829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49642fcd-501e-4621-9392-0a9135eb74bb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:50.757386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:50.826096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47927823-0b3f-4ecc-b761-713eee2504ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:50.827943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47927823-0b3f-4ecc-b761-713eee2504ef None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:50.828434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47927823-0b3f-4ecc-b761-713eee2504ef None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:50.829104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47927823-0b3f-4ecc-b761-713eee2504ef None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/13121a263c31433dbd4a5082b4bf3bb5/roles/cb4aa5570b15465eb4e81979b1019570` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:50.857576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47927823-0b3f-4ecc-b761-713eee2504ef None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=13121a263c31433dbd4a5082b4bf3bb5, role_id=cb4aa5570b15465eb4e81979b1019570)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:50.860525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47927823-0b3f-4ecc-b761-713eee2504ef None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:50.900515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 295/586] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:12:50 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/13121a263c31433dbd4a5082b4bf3bb5/roles/cb4aa5570b15465eb4e81979b1019570 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:50.901926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71896751-0fcb-407d-93b3-40e2dd5616b3 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:50.902432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47927823-0b3f-4ecc-b761-713eee2504ef None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:50.955436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc14b8-6be4-4feb-b13c-24f9ed5cd613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:50.956602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc14b8-6be4-4feb-b13c-24f9ed5cd613 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:50.957100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc14b8-6be4-4feb-b13c-24f9ed5cd613 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:50.957659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc14b8-6be4-4feb-b13c-24f9ed5cd613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:50.975516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc14b8-6be4-4feb-b13c-24f9ed5cd613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:50.978887 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc14b8-6be4-4feb-b13c-24f9ed5cd613 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-33fc14b8-6be4-4feb-b13c-24f9ed5cd613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:51.003657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:51.004642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:51.005534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:51.005534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:51.005534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:51.005534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:51.008104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 292/587] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:51.009273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc14b8-6be4-4feb-b13c-24f9ed5cd613 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:51.010041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:51.130754 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47927823-0b3f-4ecc-b761-713eee2504ef None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:51.172899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:51.203493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b333237-21b9-4335-9e79-58ef529860c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:51.204558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333237-21b9-4335-9e79-58ef529860c8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:51.205339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333237-21b9-4335-9e79-58ef529860c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:51.205807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b333237-21b9-4335-9e79-58ef529860c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:51.207569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b333237-21b9-4335-9e79-58ef529860c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:51.208020 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b333237-21b9-4335-9e79-58ef529860c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:51.209825 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b333237-21b9-4335-9e79-58ef529860c8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:51.220572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc14b8-6be4-4feb-b13c-24f9ed5cd613 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:51.239706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 296/588] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:51.239706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b333237-21b9-4335-9e79-58ef529860c8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:51.285723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74e732-1382-4805-9125-e8618c6705e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:51.287008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74e732-1382-4805-9125-e8618c6705e3 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:51.287523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74e732-1382-4805-9125-e8618c6705e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:51.288361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74e732-1382-4805-9125-e8618c6705e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:51.290927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74e732-1382-4805-9125-e8618c6705e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:51.291494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74e732-1382-4805-9125-e8618c6705e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:51.295697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74e732-1382-4805-9125-e8618c6705e3 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:51.326681 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 293/589] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:51.328253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74e732-1382-4805-9125-e8618c6705e3 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:51.416754 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:51.429306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b333237-21b9-4335-9e79-58ef529860c8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:51.482348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840bd59-2e72-4874-ac1e-91e569b5fbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:51.483499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840bd59-2e72-4874-ac1e-91e569b5fbbe None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:51.483977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840bd59-2e72-4874-ac1e-91e569b5fbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:51.484670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840bd59-2e72-4874-ac1e-91e569b5fbbe None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:51.486046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840bd59-2e72-4874-ac1e-91e569b5fbbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:51.492730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840bd59-2e72-4874-ac1e-91e569b5fbbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:51.492730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840bd59-2e72-4874-ac1e-91e569b5fbbe None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:51.516652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 297/590] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:51.516652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840bd59-2e72-4874-ac1e-91e569b5fbbe None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:51.522607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74e732-1382-4805-9125-e8618c6705e3 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:51.563448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22340a-2d50-4563-94dc-817db84bd65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:51.564693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22340a-2d50-4563-94dc-817db84bd65e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:51.565213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22340a-2d50-4563-94dc-817db84bd65e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:51.565812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22340a-2d50-4563-94dc-817db84bd65e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:51.575170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22340a-2d50-4563-94dc-817db84bd65e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:51.578774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:51.579581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22340a-2d50-4563-94dc-817db84bd65e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-9a22340a-2d50-4563-94dc-817db84bd65e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:51.607471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:51.609196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:51.610900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:51.610900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:51.610900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:51.610900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:51.610900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:51.610900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:51.610900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:51.610900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:51.620628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 294/591] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:51.622721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22340a-2d50-4563-94dc-817db84bd65e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:51.710212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840bd59-2e72-4874-ac1e-91e569b5fbbe None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:51.761202 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22340a-2d50-4563-94dc-817db84bd65e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:51.772516 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02374b67-6afc-403f-8d46-162c0fd958c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:51.773790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02374b67-6afc-403f-8d46-162c0fd958c5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:51.774273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02374b67-6afc-403f-8d46-162c0fd958c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:51.775205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02374b67-6afc-403f-8d46-162c0fd958c5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:51.785578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02374b67-6afc-403f-8d46-162c0fd958c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:51.787194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02374b67-6afc-403f-8d46-162c0fd958c5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:51.810628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d6fafb-39e2-4464-a8b5-a7cd89e057d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:51.811445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6fafb-39e2-4464-a8b5-a7cd89e057d4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:51.811725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6fafb-39e2-4464-a8b5-a7cd89e057d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:51.811998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d6fafb-39e2-4464-a8b5-a7cd89e057d4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-02374b67-6afc-403f-8d46-162c0fd958c5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:51.812812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:51.814495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:51.816280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:51.816280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:51.816280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:51.816280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:51.816280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:51.816280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:51.816280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:51.816280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:51.816280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 15:12:51.816280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:51.816280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 298/592] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:51.819081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02374b67-6afc-403f-8d46-162c0fd958c5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:51.829715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6fafb-39e2-4464-a8b5-a7cd89e057d4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:51.832918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d6fafb-39e2-4464-a8b5-a7cd89e057d4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-78d6fafb-39e2-4464-a8b5-a7cd89e057d4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:51.861823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:51.863739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:51.864953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:51.864953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:51.864953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:51.864953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:51.864953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:51.864953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:51.864953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:51.864953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:51.864953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 15:12:51.864953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:51.865980 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:51.866068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 295/593] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:51.866831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51ed0-0ecb-4a7f-bfed-e99f0cf4af34 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:51.866831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51ed0-0ecb-4a7f-bfed-e99f0cf4af34 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:51.867340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51ed0-0ecb-4a7f-bfed-e99f0cf4af34 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:51.874602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 296/594] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:51.874602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a51ed0-0ecb-4a7f-bfed-e99f0cf4af34 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:51.973644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:51.982819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02374b67-6afc-403f-8d46-162c0fd958c5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:52.046381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ab30f-ca59-462d-a2a2-97743276bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:52.047089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ab30f-ca59-462d-a2a2-97743276bd3d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:52.047375 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ab30f-ca59-462d-a2a2-97743276bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:52.047651 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ab30f-ca59-462d-a2a2-97743276bd3d None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:52.048445 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ab30f-ca59-462d-a2a2-97743276bd3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:52.048756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ab30f-ca59-462d-a2a2-97743276bd3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:52.050227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ab30f-ca59-462d-a2a2-97743276bd3d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:52.056690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a51ed0-0ecb-4a7f-bfed-e99f0cf4af34 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:52.074753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 299/595] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:52.074753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ab30f-ca59-462d-a2a2-97743276bd3d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:52.120128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7f564-6997-4b12-a781-3c78a4f2f314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:52.121553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7f564-6997-4b12-a781-3c78a4f2f314 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:52.121721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7f564-6997-4b12-a781-3c78a4f2f314 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:52.121968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7f564-6997-4b12-a781-3c78a4f2f314 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:52.122785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7f564-6997-4b12-a781-3c78a4f2f314 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:52.123215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7f564-6997-4b12-a781-3c78a4f2f314 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:52.125448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7f564-6997-4b12-a781-3c78a4f2f314 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:52.152502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 297/596] 10.222.0.93 () {56 vars in 1137 bytes} [Fri Jun 3 15:12:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:52.154650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c7bc9-fc59-490d-a727-aab705c67c23 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:52.155090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c7bc9-fc59-490d-a727-aab705c67c23 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:52.155511 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70c7bc9-fc59-490d-a727-aab705c67c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:52.177400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-b70c7bc9-fc59-490d-a727-aab705c67c23 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:52.179073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b70c7bc9-fc59-490d-a727-aab705c67c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:52.214354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:52.253671 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ab30f-ca59-462d-a2a2-97743276bd3d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:52.286227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 298/597] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:52.287220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c7bc9-fc59-490d-a727-aab705c67c23 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:52.311632 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56d7d1-cbac-4aa8-830c-1b922d7a5cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:52.313584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56d7d1-cbac-4aa8-830c-1b922d7a5cfe None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:52.314345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56d7d1-cbac-4aa8-830c-1b922d7a5cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:52.314811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56d7d1-cbac-4aa8-830c-1b922d7a5cfe None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:52.315802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56d7d1-cbac-4aa8-830c-1b922d7a5cfe None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:52.316283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56d7d1-cbac-4aa8-830c-1b922d7a5cfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:52.317873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56d7d1-cbac-4aa8-830c-1b922d7a5cfe None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:52.342892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 300/598] 10.222.0.93 () {56 vars in 1139 bytes} [Fri Jun 3 15:12:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:52.347520 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56d7d1-cbac-4aa8-830c-1b922d7a5cfe None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:52.359588 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:52.422609 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70c7bc9-fc59-490d-a727-aab705c67c23 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:52.468890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92eb1a9-5369-4da0-b71f-fcb58ba4659c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:52.469773 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92eb1a9-5369-4da0-b71f-fcb58ba4659c None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:52.470054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92eb1a9-5369-4da0-b71f-fcb58ba4659c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:52.470329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92eb1a9-5369-4da0-b71f-fcb58ba4659c None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/c1966524091242e7a6cd2758027a0d4f/roles/cb4aa5570b15465eb4e81979b1019570` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:52.486958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92eb1a9-5369-4da0-b71f-fcb58ba4659c None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=c1966524091242e7a6cd2758027a0d4f, role_id=cb4aa5570b15465eb4e81979b1019570)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:52.488424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92eb1a9-5369-4da0-b71f-fcb58ba4659c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:52.496631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56d7d1-cbac-4aa8-830c-1b922d7a5cfe None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:52.522909 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 299/599] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:12:52 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/c1966524091242e7a6cd2758027a0d4f/roles/cb4aa5570b15465eb4e81979b1019570 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:52.524357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92eb1a9-5369-4da0-b71f-fcb58ba4659c None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:52.551603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7464608-19f3-43a5-a18e-8ad421174f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:52.553005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7464608-19f3-43a5-a18e-8ad421174f0e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:52.553472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7464608-19f3-43a5-a18e-8ad421174f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:52.553936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7464608-19f3-43a5-a18e-8ad421174f0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:52.559596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:52.575566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7464608-19f3-43a5-a18e-8ad421174f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:52.578309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7464608-19f3-43a5-a18e-8ad421174f0e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f7464608-19f3-43a5-a18e-8ad421174f0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:52.618735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:52.619655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:52.620621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:52.620621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:52.620621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:52.620621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:52.622433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 301/600] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:52.625913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7464608-19f3-43a5-a18e-8ad421174f0e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:52.739014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:52.753566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92eb1a9-5369-4da0-b71f-fcb58ba4659c None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:52.814028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47d532-1aaa-4dc1-a689-cb759cf0ea3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:52.815433 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47d532-1aaa-4dc1-a689-cb759cf0ea3c None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:52.815896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47d532-1aaa-4dc1-a689-cb759cf0ea3c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:52.816286 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47d532-1aaa-4dc1-a689-cb759cf0ea3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:52.829336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec47d532-1aaa-4dc1-a689-cb759cf0ea3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:52.830397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7464608-19f3-43a5-a18e-8ad421174f0e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ec47d532-1aaa-4dc1-a689-cb759cf0ea3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:52.837225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:52.838172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:52.839077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:52.842917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 300/601] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:52.843502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47d532-1aaa-4dc1-a689-cb759cf0ea3c None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:52.884858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f270a-b98e-4999-bf4f-4bf240feb006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:52.885986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f270a-b98e-4999-bf4f-4bf240feb006 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:52.886414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f270a-b98e-4999-bf4f-4bf240feb006 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:52.886994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f270a-b98e-4999-bf4f-4bf240feb006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:52.904750 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f270a-b98e-4999-bf4f-4bf240feb006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:52.907193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281f270a-b98e-4999-bf4f-4bf240feb006 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-281f270a-b98e-4999-bf4f-4bf240feb006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:52.947321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:52.949448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:52.951076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:52.951076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:52.951076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:52.951076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:52.951076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 302/602] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:52.957770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f270a-b98e-4999-bf4f-4bf240feb006 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:53.006029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47d532-1aaa-4dc1-a689-cb759cf0ea3c None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:53.043589 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:53.052192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ceb3c-9295-4f04-950b-6bbbd08d191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:53.052192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ceb3c-9295-4f04-950b-6bbbd08d191a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:53.052192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ceb3c-9295-4f04-950b-6bbbd08d191a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:53.052192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ceb3c-9295-4f04-950b-6bbbd08d191a None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:53.052847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ceb3c-9295-4f04-950b-6bbbd08d191a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:53.052847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ceb3c-9295-4f04-950b-6bbbd08d191a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:53.054734 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ceb3c-9295-4f04-950b-6bbbd08d191a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:53.072490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 301/603] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:53.074680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ceb3c-9295-4f04-950b-6bbbd08d191a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:53.094952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:53.180305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f270a-b98e-4999-bf4f-4bf240feb006 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:53.244287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c12514-5cb0-4017-b548-bb0ab116bb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:53.246041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c12514-5cb0-4017-b548-bb0ab116bb06 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:53.246834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c12514-5cb0-4017-b548-bb0ab116bb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:53.247554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c12514-5cb0-4017-b548-bb0ab116bb06 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:53.249176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c12514-5cb0-4017-b548-bb0ab116bb06 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:53.249952 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c12514-5cb0-4017-b548-bb0ab116bb06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:53.252758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c12514-5cb0-4017-b548-bb0ab116bb06 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:53.254480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ceb3c-9295-4f04-950b-6bbbd08d191a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:53.271233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 303/604] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:53.273103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c12514-5cb0-4017-b548-bb0ab116bb06 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:53.330128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdab50-2230-4b17-a773-dc278e16c141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:53.331033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdab50-2230-4b17-a773-dc278e16c141 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:53.331522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdab50-2230-4b17-a773-dc278e16c141 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:53.332010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdab50-2230-4b17-a773-dc278e16c141 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:53.333341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdab50-2230-4b17-a773-dc278e16c141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:53.333883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdab50-2230-4b17-a773-dc278e16c141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:53.336693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdab50-2230-4b17-a773-dc278e16c141 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:53.365112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 302/605] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:53.366675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdab50-2230-4b17-a773-dc278e16c141 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:53.459249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:53.465155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c12514-5cb0-4017-b548-bb0ab116bb06 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:53.503348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc917a-5f05-4419-b255-2d20faf4a39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:53.504849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc917a-5f05-4419-b255-2d20faf4a39f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:53.505353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc917a-5f05-4419-b255-2d20faf4a39f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:53.505801 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc917a-5f05-4419-b255-2d20faf4a39f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:53.510566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:53.517100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc917a-5f05-4419-b255-2d20faf4a39f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:53.518838 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc917a-5f05-4419-b255-2d20faf4a39f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-39fc917a-5f05-4419-b255-2d20faf4a39f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:53.537254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:53.539448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:53.541562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:53.541562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:53.541562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:53.541562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:53.541562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:53.541562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:53.541562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:53.541562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:53.541562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 304/606] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:53.542660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc917a-5f05-4419-b255-2d20faf4a39f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:53.567921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdab50-2230-4b17-a773-dc278e16c141 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:53.632149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f000053-775c-4fea-b0e2-1793edcf2bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:53.633800 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f000053-775c-4fea-b0e2-1793edcf2bcc None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:53.634580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f000053-775c-4fea-b0e2-1793edcf2bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:53.635277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f000053-775c-4fea-b0e2-1793edcf2bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:53.660981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f000053-775c-4fea-b0e2-1793edcf2bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:53.664246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f000053-775c-4fea-b0e2-1793edcf2bcc None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:53.665205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc917a-5f05-4419-b255-2d20faf4a39f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-2f000053-775c-4fea-b0e2-1793edcf2bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:53.698399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:53.700799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:53.702737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:53.702737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:53.702737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:53.702737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:53.712964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453dfe2a-da6e-4ce3-9f50-344c8e27f4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:53.713218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 303/607] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:53.713370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dfe2a-da6e-4ce3-9f50-344c8e27f4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:53.713370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dfe2a-da6e-4ce3-9f50-344c8e27f4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:53.713370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453dfe2a-da6e-4ce3-9f50-344c8e27f4f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:53.713718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f000053-775c-4fea-b0e2-1793edcf2bcc None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:53.714873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453dfe2a-da6e-4ce3-9f50-344c8e27f4f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:53.716097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453dfe2a-da6e-4ce3-9f50-344c8e27f4f4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-453dfe2a-da6e-4ce3-9f50-344c8e27f4f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:53.731346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:53.732267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:53.733184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:53.733184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:53.733184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:53.733184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:53.733184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:53.733184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:53.733184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:53.733184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:53.733184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 305/608] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:53.734301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e86cd-71d3-4653-84cf-6b5cd49a05b3 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:53.734301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e86cd-71d3-4653-84cf-6b5cd49a05b3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:53.734301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994e86cd-71d3-4653-84cf-6b5cd49a05b3 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:53.735640 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 306/609] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:53.736148 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e86cd-71d3-4653-84cf-6b5cd49a05b3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:53.761470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:53.832463 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f000053-775c-4fea-b0e2-1793edcf2bcc None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:53.875307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:53.879737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994e86cd-71d3-4653-84cf-6b5cd49a05b3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:53.889110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f0114-3255-4229-8101-6b8e2074639d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:53.889841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f0114-3255-4229-8101-6b8e2074639d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:53.890122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f0114-3255-4229-8101-6b8e2074639d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:53.890453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f0114-3255-4229-8101-6b8e2074639d None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:53.891528 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f0114-3255-4229-8101-6b8e2074639d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:53.891715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f0114-3255-4229-8101-6b8e2074639d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:53.893081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f0114-3255-4229-8101-6b8e2074639d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:53.914301 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 304/610] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:53.914301 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f0114-3255-4229-8101-6b8e2074639d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:53.938383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77eecb-3b97-44ef-80ec-d11cc6ffb026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:53.939169 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77eecb-3b97-44ef-80ec-d11cc6ffb026 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:53.939461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77eecb-3b97-44ef-80ec-d11cc6ffb026 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:53.939793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77eecb-3b97-44ef-80ec-d11cc6ffb026 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:53.940517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77eecb-3b97-44ef-80ec-d11cc6ffb026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:53.940819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77eecb-3b97-44ef-80ec-d11cc6ffb026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:53.942221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77eecb-3b97-44ef-80ec-d11cc6ffb026 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:53.954899 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 307/611] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:53.955979 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158f02e-157d-4318-be2b-a8b7f251dba7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:53.956263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158f02e-157d-4318-be2b-a8b7f251dba7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:53.956535 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158f02e-157d-4318-be2b-a8b7f251dba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:53.976997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f158f02e-157d-4318-be2b-a8b7f251dba7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:53.978733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f158f02e-157d-4318-be2b-a8b7f251dba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:54.039405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f0114-3255-4229-8101-6b8e2074639d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:54.055502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 308/612] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:54.063805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158f02e-157d-4318-be2b-a8b7f251dba7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:54.078568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:54.095723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8755f-7c9a-4537-bf81-d5c995608dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:54.096410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8755f-7c9a-4537-bf81-d5c995608dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:54.096688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8755f-7c9a-4537-bf81-d5c995608dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:54.096959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8755f-7c9a-4537-bf81-d5c995608dc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:54.097757 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8755f-7c9a-4537-bf81-d5c995608dc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:54.098065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8755f-7c9a-4537-bf81-d5c995608dc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:54.099569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca8755f-7c9a-4537-bf81-d5c995608dc6 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:54.112708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 305/613] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:54.115411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8755f-7c9a-4537-bf81-d5c995608dc6 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:54.175382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:54.184245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158f02e-157d-4318-be2b-a8b7f251dba7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:54.223748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72362fcb-9d2c-4302-92a1-8000e0b2a5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:54.224631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72362fcb-9d2c-4302-92a1-8000e0b2a5b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:54.224908 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72362fcb-9d2c-4302-92a1-8000e0b2a5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:54.225182 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72362fcb-9d2c-4302-92a1-8000e0b2a5b4 None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/98aa51c4bc604b0ca8a6c69bc1455bbb/roles/cb4aa5570b15465eb4e81979b1019570` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:54.242791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8755f-7c9a-4537-bf81-d5c995608dc6 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:54.253709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72362fcb-9d2c-4302-92a1-8000e0b2a5b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=98aa51c4bc604b0ca8a6c69bc1455bbb, role_id=cb4aa5570b15465eb4e81979b1019570)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:54.255844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72362fcb-9d2c-4302-92a1-8000e0b2a5b4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:54.273067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 309/614] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:12:54 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/98aa51c4bc604b0ca8a6c69bc1455bbb/roles/cb4aa5570b15465eb4e81979b1019570 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:54.273898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72362fcb-9d2c-4302-92a1-8000e0b2a5b4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:54.316964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d486cf-61d7-4a59-84f6-bf5c1ea46c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:54.319224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d486cf-61d7-4a59-84f6-bf5c1ea46c6e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:54.319721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d486cf-61d7-4a59-84f6-bf5c1ea46c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:54.320246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d486cf-61d7-4a59-84f6-bf5c1ea46c6e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:54.334000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d486cf-61d7-4a59-84f6-bf5c1ea46c6e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:54.337206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d486cf-61d7-4a59-84f6-bf5c1ea46c6e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-c0d486cf-61d7-4a59-84f6-bf5c1ea46c6e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:54.361190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:54.362212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:54.363185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:54.363185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:54.363185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:54.363185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:54.363185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:54.363185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:54.363185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:54.363185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:54.363185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:12:54.363185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:54.373158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 306/615] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:12:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:54.373158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d486cf-61d7-4a59-84f6-bf5c1ea46c6e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:54.384274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:54.447730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72362fcb-9d2c-4302-92a1-8000e0b2a5b4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:54.494247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c8d10-bc98-46c3-84cb-76ae87c67e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:54.495568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c8d10-bc98-46c3-84cb-76ae87c67e35 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:54.496619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c8d10-bc98-46c3-84cb-76ae87c67e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:54.497105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c8d10-bc98-46c3-84cb-76ae87c67e35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:54.517662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7c8d10-bc98-46c3-84cb-76ae87c67e35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:54.523510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d486cf-61d7-4a59-84f6-bf5c1ea46c6e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2f7c8d10-bc98-46c3-84cb-76ae87c67e35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:54.531527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:54.533193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 310/616] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:54.535583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c8d10-bc98-46c3-84cb-76ae87c67e35 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:54.539801 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:54.577040 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb23192-c55d-43ca-ab3e-79db895fc4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:54.577940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb23192-c55d-43ca-ab3e-79db895fc4d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:54.578218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb23192-c55d-43ca-ab3e-79db895fc4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:54.578485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb23192-c55d-43ca-ab3e-79db895fc4d5 None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/e3f887d5bbcd424884537b95affd96f8/roles/cb4aa5570b15465eb4e81979b1019570` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:54.598147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb23192-c55d-43ca-ab3e-79db895fc4d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=e3f887d5bbcd424884537b95affd96f8, role_id=cb4aa5570b15465eb4e81979b1019570)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:54.599649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb23192-c55d-43ca-ab3e-79db895fc4d5 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:54.627020 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 307/617] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:12:54 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/e3f887d5bbcd424884537b95affd96f8/roles/cb4aa5570b15465eb4e81979b1019570 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:54.631919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb23192-c55d-43ca-ab3e-79db895fc4d5 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:54.732275 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c8d10-bc98-46c3-84cb-76ae87c67e35 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:54.776463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:54.801389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb23192-c55d-43ca-ab3e-79db895fc4d5 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:54.821115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f8460-2f08-40cb-862b-cc282784b113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:54.822015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f8460-2f08-40cb-862b-cc282784b113 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:54.822304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f8460-2f08-40cb-862b-cc282784b113 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:54.822603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f8460-2f08-40cb-862b-cc282784b113 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:54.823465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f8460-2f08-40cb-862b-cc282784b113 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:54.823819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f8460-2f08-40cb-862b-cc282784b113 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:54.825542 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f8460-2f08-40cb-862b-cc282784b113 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:54.870005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 311/618] 10.222.0.93 () {56 vars in 1135 bytes} [Fri Jun 3 15:12:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:54.871775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc658a-aa9a-4f3e-93ef-3a0af4acdd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:54.872468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc658a-aa9a-4f3e-93ef-3a0af4acdd39 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:54.872751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc658a-aa9a-4f3e-93ef-3a0af4acdd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:54.873022 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc658a-aa9a-4f3e-93ef-3a0af4acdd39 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:54.873797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc658a-aa9a-4f3e-93ef-3a0af4acdd39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:54.874109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc658a-aa9a-4f3e-93ef-3a0af4acdd39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:54.876230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f8460-2f08-40cb-862b-cc282784b113 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:54.877468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc658a-aa9a-4f3e-93ef-3a0af4acdd39 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:54.898856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 308/619] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:54.905720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc658a-aa9a-4f3e-93ef-3a0af4acdd39 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:54.968050 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:55.003006 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f8460-2f08-40cb-862b-cc282784b113 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:55.058918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69435c-48ee-4574-8b15-e480e7a2eaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:55.060274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69435c-48ee-4574-8b15-e480e7a2eaa2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:55.060846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69435c-48ee-4574-8b15-e480e7a2eaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:55.061405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69435c-48ee-4574-8b15-e480e7a2eaa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:55.081898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69435c-48ee-4574-8b15-e480e7a2eaa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:55.083652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69435c-48ee-4574-8b15-e480e7a2eaa2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:55.090884 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc658a-aa9a-4f3e-93ef-3a0af4acdd39 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-3b69435c-48ee-4574-8b15-e480e7a2eaa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:55.109420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:55.110417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:55.111339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:55.111339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:55.111339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:55.111339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:55.111339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 312/620] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:55.116865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69435c-48ee-4574-8b15-e480e7a2eaa2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:55.146854 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16699c59-ffd3-41b2-84d6-e2cdad43867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:55.148025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16699c59-ffd3-41b2-84d6-e2cdad43867e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:55.148513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16699c59-ffd3-41b2-84d6-e2cdad43867e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:55.149017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16699c59-ffd3-41b2-84d6-e2cdad43867e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:55.172574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16699c59-ffd3-41b2-84d6-e2cdad43867e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:55.175061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16699c59-ffd3-41b2-84d6-e2cdad43867e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:55.181257 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-16699c59-ffd3-41b2-84d6-e2cdad43867e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:55.209631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:55.210851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:55.212103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:55.212103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:55.212103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:55.212103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:55.212103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 309/621] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:12:55.219059 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16699c59-ffd3-41b2-84d6-e2cdad43867e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:55.302874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:55.319926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69435c-48ee-4574-8b15-e480e7a2eaa2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:55.339401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16699c59-ffd3-41b2-84d6-e2cdad43867e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:55.378720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41779eab-36bf-4260-8ba6-07c43290b5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:55.379540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779eab-36bf-4260-8ba6-07c43290b5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:55.379920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779eab-36bf-4260-8ba6-07c43290b5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:55.380295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41779eab-36bf-4260-8ba6-07c43290b5b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:55.380598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d376c7-369f-4636-9fa1-5068020fe58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:55.381242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41779eab-36bf-4260-8ba6-07c43290b5b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:55.381651 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d376c7-369f-4636-9fa1-5068020fe58c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:55.381767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41779eab-36bf-4260-8ba6-07c43290b5b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:55.382191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d376c7-369f-4636-9fa1-5068020fe58c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:55.382783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d376c7-369f-4636-9fa1-5068020fe58c None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:55.383431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41779eab-36bf-4260-8ba6-07c43290b5b0 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:55.384086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d376c7-369f-4636-9fa1-5068020fe58c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:55.384537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d376c7-369f-4636-9fa1-5068020fe58c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:55.387224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d376c7-369f-4636-9fa1-5068020fe58c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:55.397768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 310/622] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:55.401431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d5041-95dd-469e-8b8b-1c6022f047c6 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:55.401951 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d5041-95dd-469e-8b8b-1c6022f047c6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:55.402331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d5041-95dd-469e-8b8b-1c6022f047c6 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:55.405044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 311/623] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:55.405912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d5041-95dd-469e-8b8b-1c6022f047c6 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:55.408696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 313/624] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:55.410149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5f081-59ab-42be-adde-12b9028667e5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:55.411277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5f081-59ab-42be-adde-12b9028667e5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:55.411663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5f081-59ab-42be-adde-12b9028667e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:55.430460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-21f5f081-59ab-42be-adde-12b9028667e5 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:55.431814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-21f5f081-59ab-42be-adde-12b9028667e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:55.543330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 314/625] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:55.544582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5f081-59ab-42be-adde-12b9028667e5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:55.571141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d5041-95dd-469e-8b8b-1c6022f047c6 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:55.590010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:55.615434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:55.622903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fce324-0dd0-4d73-a696-92c7ad274a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:55.623986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fce324-0dd0-4d73-a696-92c7ad274a3c None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:55.623986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fce324-0dd0-4d73-a696-92c7ad274a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:55.624885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fce324-0dd0-4d73-a696-92c7ad274a3c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:55.632150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fce324-0dd0-4d73-a696-92c7ad274a3c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:55.634104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fce324-0dd0-4d73-a696-92c7ad274a3c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-20fce324-0dd0-4d73-a696-92c7ad274a3c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:55.655494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:55.657150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 312/626] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:55.658456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fce324-0dd0-4d73-a696-92c7ad274a3c None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:55.732541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5f081-59ab-42be-adde-12b9028667e5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:55.782664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b66743-9653-40e2-9434-e3fd3412647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:55.784368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b66743-9653-40e2-9434-e3fd3412647f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:55.785010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b66743-9653-40e2-9434-e3fd3412647f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:55.785650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b66743-9653-40e2-9434-e3fd3412647f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:55.798149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b66743-9653-40e2-9434-e3fd3412647f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:55.801183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b66743-9653-40e2-9434-e3fd3412647f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-65b66743-9653-40e2-9434-e3fd3412647f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:55.821419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:55.822373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:55.823537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:55.823537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:55.823537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:55.823537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:55.823537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:55.823537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:55.823537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:55.823537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:55.825610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fce324-0dd0-4d73-a696-92c7ad274a3c None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:55.826793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 315/627] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:55.827896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b66743-9653-40e2-9434-e3fd3412647f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:55.868495 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735291c9-3c4d-4e30-9a9b-6f7ac16f0450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:55.869619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735291c9-3c4d-4e30-9a9b-6f7ac16f0450 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:55.870088 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735291c9-3c4d-4e30-9a9b-6f7ac16f0450 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:55.871119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735291c9-3c4d-4e30-9a9b-6f7ac16f0450 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:55.884539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735291c9-3c4d-4e30-9a9b-6f7ac16f0450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-735291c9-3c4d-4e30-9a9b-6f7ac16f0450 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:55.894099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:55.895075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:55.895940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:55.902782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 313/628] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:55.903901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735291c9-3c4d-4e30-9a9b-6f7ac16f0450 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:55.946367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:55.969619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:55.980894 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b66743-9653-40e2-9434-e3fd3412647f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:56.030301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb31d-772d-4cc8-bcea-0a2cdb78d821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:56.031002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb31d-772d-4cc8-bcea-0a2cdb78d821 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:56.031307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb31d-772d-4cc8-bcea-0a2cdb78d821 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:56.031649 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb31d-772d-4cc8-bcea-0a2cdb78d821 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:56.032781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eb31d-772d-4cc8-bcea-0a2cdb78d821 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:56.033112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eb31d-772d-4cc8-bcea-0a2cdb78d821 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:56.035346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eb31d-772d-4cc8-bcea-0a2cdb78d821 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:56.041428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735291c9-3c4d-4e30-9a9b-6f7ac16f0450 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:56.066416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 316/629] 10.222.0.93 () {56 vars in 1137 bytes} [Fri Jun 3 15:12:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:56.066416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb31d-772d-4cc8-bcea-0a2cdb78d821 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:56.090058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffd44c-8107-481c-9596-0d0d17108033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:56.091564 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffd44c-8107-481c-9596-0d0d17108033 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:56.092068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffd44c-8107-481c-9596-0d0d17108033 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:56.092867 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfffd44c-8107-481c-9596-0d0d17108033 None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:56.094048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfffd44c-8107-481c-9596-0d0d17108033 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:56.095235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfffd44c-8107-481c-9596-0d0d17108033 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:56.098174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfffd44c-8107-481c-9596-0d0d17108033 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:56.118256 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 314/630] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:56.119546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffd44c-8107-481c-9596-0d0d17108033 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:56.202626 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb31d-772d-4cc8-bcea-0a2cdb78d821 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:56.246649 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e32af4-e655-4b43-8a75-14a57c76e607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:56.247672 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e32af4-e655-4b43-8a75-14a57c76e607 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:56.248030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e32af4-e655-4b43-8a75-14a57c76e607 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:56.248381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e32af4-e655-4b43-8a75-14a57c76e607 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:56.249209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e32af4-e655-4b43-8a75-14a57c76e607 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:56.249588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e32af4-e655-4b43-8a75-14a57c76e607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:56.251089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e32af4-e655-4b43-8a75-14a57c76e607 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:56.251675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfffd44c-8107-481c-9596-0d0d17108033 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:56.257136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:56.263661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:56.266265 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 317/631] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:56.268397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e32af4-e655-4b43-8a75-14a57c76e607 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:56.291858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e8e81-f542-4e7a-9806-116f8076b8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:56.293041 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e8e81-f542-4e7a-9806-116f8076b8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:56.293325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e8e81-f542-4e7a-9806-116f8076b8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:56.293598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e8e81-f542-4e7a-9806-116f8076b8a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:56.365340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e8e81-f542-4e7a-9806-116f8076b8a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:56.366829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e8e81-f542-4e7a-9806-116f8076b8a8 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:56.392744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e32af4-e655-4b43-8a75-14a57c76e607 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:56.441961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c444558-b2ed-404f-bd4f-82dd3876830c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:56.443295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c444558-b2ed-404f-bd4f-82dd3876830c None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:56.443929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c444558-b2ed-404f-bd4f-82dd3876830c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:56.444412 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c444558-b2ed-404f-bd4f-82dd3876830c None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/bb4fb8f859dc4489b8c4fd8fdc09c3f6/roles/cb4aa5570b15465eb4e81979b1019570` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:56.466680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c444558-b2ed-404f-bd4f-82dd3876830c None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=bb4fb8f859dc4489b8c4fd8fdc09c3f6, role_id=cb4aa5570b15465eb4e81979b1019570)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:56.468686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c444558-b2ed-404f-bd4f-82dd3876830c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:56.479893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 315/632] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:56.480854 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e8e81-f542-4e7a-9806-116f8076b8a8 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:56.487316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 318/633] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:12:56 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/bb4fb8f859dc4489b8c4fd8fdc09c3f6/roles/cb4aa5570b15465eb4e81979b1019570 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:12:56.488549 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c444558-b2ed-404f-bd4f-82dd3876830c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:56.494218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:56.557608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:56.630130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c444558-b2ed-404f-bd4f-82dd3876830c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:56.670999 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e8e81-f542-4e7a-9806-116f8076b8a8 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:56.676111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff2f7a-f32d-4f5c-95fd-e64aabfa0e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:56.677191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff2f7a-f32d-4f5c-95fd-e64aabfa0e66 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:56.677680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff2f7a-f32d-4f5c-95fd-e64aabfa0e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:56.678156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff2f7a-f32d-4f5c-95fd-e64aabfa0e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:56.707869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff2f7a-f32d-4f5c-95fd-e64aabfa0e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:56.712293 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff2f7a-f32d-4f5c-95fd-e64aabfa0e66 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b2ff2f7a-f32d-4f5c-95fd-e64aabfa0e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:56.740252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:56.741215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:56.742230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:56.742230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:56.742230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:56.742230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:56.742474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 319/634] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:56.749373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff2f7a-f32d-4f5c-95fd-e64aabfa0e66 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:56.751799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d9f60-5234-49e4-ab40-f9cf4a180390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:56.752664 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d9f60-5234-49e4-ab40-f9cf4a180390 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:56.752948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d9f60-5234-49e4-ab40-f9cf4a180390 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:56.753220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d9f60-5234-49e4-ab40-f9cf4a180390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:56.767920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d9f60-5234-49e4-ab40-f9cf4a180390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:56.769389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d9f60-5234-49e4-ab40-f9cf4a180390 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-611d9f60-5234-49e4-ab40-f9cf4a180390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:56.790697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:56.791678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:56.792611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:56.792611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:56.792611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:56.792611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:56.792611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 316/635] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:56.797771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d9f60-5234-49e4-ab40-f9cf4a180390 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:56.869620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff2f7a-f32d-4f5c-95fd-e64aabfa0e66 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:56.871878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:56.911964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:56.914140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf019ecc-b8ef-4754-bc42-fa6f0a123230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:56.915164 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf019ecc-b8ef-4754-bc42-fa6f0a123230 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:56.915671 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf019ecc-b8ef-4754-bc42-fa6f0a123230 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:56.916148 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf019ecc-b8ef-4754-bc42-fa6f0a123230 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:56.917206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf019ecc-b8ef-4754-bc42-fa6f0a123230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:56.918506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf019ecc-b8ef-4754-bc42-fa6f0a123230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:56.920127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf019ecc-b8ef-4754-bc42-fa6f0a123230 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:56.932822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 320/636] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:56.940011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf019ecc-b8ef-4754-bc42-fa6f0a123230 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:56.974569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d9f60-5234-49e4-ab40-f9cf4a180390 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:57.003265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cca4e-cb7e-47a8-98b6-6aae364fbaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:57.004204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cca4e-cb7e-47a8-98b6-6aae364fbaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:57.004716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cca4e-cb7e-47a8-98b6-6aae364fbaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:57.005218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9cca4e-cb7e-47a8-98b6-6aae364fbaf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:57.006784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cca4e-cb7e-47a8-98b6-6aae364fbaf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:57.007359 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cca4e-cb7e-47a8-98b6-6aae364fbaf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:57.009902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9cca4e-cb7e-47a8-98b6-6aae364fbaf2 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:57.026744 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 317/637] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:12:57.032874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cca4e-cb7e-47a8-98b6-6aae364fbaf2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:57.179394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9cca4e-cb7e-47a8-98b6-6aae364fbaf2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:57.200565 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf019ecc-b8ef-4754-bc42-fa6f0a123230 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:57.206027 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:57.218943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43bf9c-b5f1-4307-8fdf-a53697f76ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:57.219741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43bf9c-b5f1-4307-8fdf-a53697f76ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:57.220063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43bf9c-b5f1-4307-8fdf-a53697f76ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:57.220332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43bf9c-b5f1-4307-8fdf-a53697f76ce0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:57.229756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43bf9c-b5f1-4307-8fdf-a53697f76ce0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:57.231264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43bf9c-b5f1-4307-8fdf-a53697f76ce0 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ad43bf9c-b5f1-4307-8fdf-a53697f76ce0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:57.247850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:57.249535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:57.250504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:57.250504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:57.250504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:57.250504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:57.250504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:57.250504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:57.250504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:57.250504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:57.250504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:12:57.250504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:57.250504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 318/638] 10.222.0.93 () {56 vars in 1093 bytes} [Fri Jun 3 15:12:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:57.251623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff449610-ca28-4e09-9672-fc294bd4ddd1 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:57.251981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff449610-ca28-4e09-9672-fc294bd4ddd1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:57.252357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff449610-ca28-4e09-9672-fc294bd4ddd1 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:57.254057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 319/639] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:57.254439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fa860-92f8-4477-91c4-861f093112d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:57.254789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff449610-ca28-4e09-9672-fc294bd4ddd1 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:57.256066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fa860-92f8-4477-91c4-861f093112d0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:57.256829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fa860-92f8-4477-91c4-861f093112d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:57.257329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fa860-92f8-4477-91c4-861f093112d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:57.266496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38fa860-92f8-4477-91c4-861f093112d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:57.268427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38fa860-92f8-4477-91c4-861f093112d0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a38fa860-92f8-4477-91c4-861f093112d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:57.284007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:57.285041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:57.285961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:57.285961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:57.285961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:57.285961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:57.285961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:57.285961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:57.285961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:57.285961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:57.288733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 321/640] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:57.290279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af2c8-10a9-4bfb-a33b-154c3f72c0be None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:57.290806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af2c8-10a9-4bfb-a33b-154c3f72c0be None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:57.291307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af2c8-10a9-4bfb-a33b-154c3f72c0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:57.310784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-285af2c8-10a9-4bfb-a33b-154c3f72c0be None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:57.312734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-285af2c8-10a9-4bfb-a33b-154c3f72c0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:57.333067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:57.400518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 322/641] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:57.402836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285af2c8-10a9-4bfb-a33b-154c3f72c0be None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:57.414497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff449610-ca28-4e09-9672-fc294bd4ddd1 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:57.458048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5119b-f4bd-44e1-bee1-273c8d99f694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:57.458679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5119b-f4bd-44e1-bee1-273c8d99f694 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:57.458957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5119b-f4bd-44e1-bee1-273c8d99f694 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:57.459235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5119b-f4bd-44e1-bee1-273c8d99f694 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:57.460286 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5119b-f4bd-44e1-bee1-273c8d99f694 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:57.460442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5119b-f4bd-44e1-bee1-273c8d99f694 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:57.461961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5119b-f4bd-44e1-bee1-273c8d99f694 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:57.483640 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 320/642] 10.222.0.93 () {56 vars in 1133 bytes} [Fri Jun 3 15:12:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:57.484860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5119b-f4bd-44e1-bee1-273c8d99f694 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:57.536149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:57.537959 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285af2c8-10a9-4bfb-a33b-154c3f72c0be None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:57.582110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a398ebe-dc02-450c-847a-29533396d603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:57.583314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a398ebe-dc02-450c-847a-29533396d603 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:57.583889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a398ebe-dc02-450c-847a-29533396d603 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:57.584370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a398ebe-dc02-450c-847a-29533396d603 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:57.585385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a398ebe-dc02-450c-847a-29533396d603 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:57.585914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a398ebe-dc02-450c-847a-29533396d603 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:57.588320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a398ebe-dc02-450c-847a-29533396d603 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:57.603289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 323/643] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:57.604477 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a398ebe-dc02-450c-847a-29533396d603 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:57.605395 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5119b-f4bd-44e1-bee1-273c8d99f694 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:57.621133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:57.646543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db8c25-d1fc-4e90-8076-e0aa057bd12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:57.647345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db8c25-d1fc-4e90-8076-e0aa057bd12f None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:57.647625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db8c25-d1fc-4e90-8076-e0aa057bd12f None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:57.647897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db8c25-d1fc-4e90-8076-e0aa057bd12f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:57.660805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db8c25-d1fc-4e90-8076-e0aa057bd12f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-d7db8c25-d1fc-4e90-8076-e0aa057bd12f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:57.669432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:57.671360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 321/644] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:57.673156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db8c25-d1fc-4e90-8076-e0aa057bd12f None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:57.757848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a398ebe-dc02-450c-847a-29533396d603 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:57.813950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd52caf-c645-47cb-9d38-33ea321b8065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:57.814167 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:57.815243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd52caf-c645-47cb-9d38-33ea321b8065 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:57.815243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd52caf-c645-47cb-9d38-33ea321b8065 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:57.815243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd52caf-c645-47cb-9d38-33ea321b8065 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:57.829558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db8c25-d1fc-4e90-8076-e0aa057bd12f None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:57.868613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd72170-95ad-4dc2-8220-3c47837fd3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:57.869287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd72170-95ad-4dc2-8220-3c47837fd3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:57.869569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd72170-95ad-4dc2-8220-3c47837fd3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:57.869840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd72170-95ad-4dc2-8220-3c47837fd3f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:57.871941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd72170-95ad-4dc2-8220-3c47837fd3f6 None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=13121a263c31433dbd4a5082b4bf3bb5) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:57.872370 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd72170-95ad-4dc2-8220-3c47837fd3f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:57.874822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd72170-95ad-4dc2-8220-3c47837fd3f6 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:57.894573 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 322/645] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:12:57 2022] GET /identity/v3/role_assignments?user.id=13121a263c31433dbd4a5082b4bf3bb5&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:57.894573 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd72170-95ad-4dc2-8220-3c47837fd3f6 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:57.940741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd52caf-c645-47cb-9d38-33ea321b8065 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:57.942276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd52caf-c645-47cb-9d38-33ea321b8065 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:58.015300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:58.066577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd72170-95ad-4dc2-8220-3c47837fd3f6 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:58.107159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 324/646] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:12:58.109769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff652e-82b0-4566-a05c-8eabd035b63e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:58.110361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff652e-82b0-4566-a05c-8eabd035b63e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:58.110922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff652e-82b0-4566-a05c-8eabd035b63e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:58.113245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 325/647] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:58.114144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff652e-82b0-4566-a05c-8eabd035b63e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:58.127912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2175f79-626b-47d7-92f9-669225af65b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:58.128817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2175f79-626b-47d7-92f9-669225af65b0 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:58.129099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2175f79-626b-47d7-92f9-669225af65b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:58.129381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2175f79-626b-47d7-92f9-669225af65b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:58.130162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2175f79-626b-47d7-92f9-669225af65b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:58.130469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2175f79-626b-47d7-92f9-669225af65b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:58.131979 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2175f79-626b-47d7-92f9-669225af65b0 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:58.144674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 323/648] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:58.146052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f12fa0-e718-4371-9bf1-d1f96c3dd13f None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:58.146348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f12fa0-e718-4371-9bf1-d1f96c3dd13f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:58.146659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f12fa0-e718-4371-9bf1-d1f96c3dd13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:58.174867 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-96f12fa0-e718-4371-9bf1-d1f96c3dd13f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:12:58.176558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-96f12fa0-e718-4371-9bf1-d1f96c3dd13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:12:58.212247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:58.287094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 324/649] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:12:58.293242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff652e-82b0-4566-a05c-8eabd035b63e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:58.295556 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f12fa0-e718-4371-9bf1-d1f96c3dd13f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:58.332852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:58.363576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea335673-2d66-477e-b051-545b823bfc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:58.364482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea335673-2d66-477e-b051-545b823bfc04 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:58.365072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea335673-2d66-477e-b051-545b823bfc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:58.365349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea335673-2d66-477e-b051-545b823bfc04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:58.379628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea335673-2d66-477e-b051-545b823bfc04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:58.382049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea335673-2d66-477e-b051-545b823bfc04 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ea335673-2d66-477e-b051-545b823bfc04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:58.410521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:58.412416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:58.413860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:58.413860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:58.413860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:58.413860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:58.413860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 326/650] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:58.413860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea335673-2d66-477e-b051-545b823bfc04 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:58.460694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f12fa0-e718-4371-9bf1-d1f96c3dd13f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:58.514386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e1d04-de4b-47a4-b4b4-cd26a21b4187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:58.516054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1d04-de4b-47a4-b4b4-cd26a21b4187 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:58.516672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1d04-de4b-47a4-b4b4-cd26a21b4187 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:58.517277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e1d04-de4b-47a4-b4b4-cd26a21b4187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:58.540951 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e1d04-de4b-47a4-b4b4-cd26a21b4187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:58.543584 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e1d04-de4b-47a4-b4b4-cd26a21b4187 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-977e1d04-de4b-47a4-b4b4-cd26a21b4187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:58.564752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:58.565910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:58.567096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:58.567096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:58.567096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:58.567096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:58.568203 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea335673-2d66-477e-b051-545b823bfc04 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:58.573706 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 325/651] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:58.574861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e1d04-de4b-47a4-b4b4-cd26a21b4187 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:58.588868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:58.620719 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f82dc-d41f-4aaa-aaf2-becb170bdf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:58.621812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f82dc-d41f-4aaa-aaf2-becb170bdf5c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:58.622239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f82dc-d41f-4aaa-aaf2-becb170bdf5c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:58.622881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f82dc-d41f-4aaa-aaf2-becb170bdf5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:58.642973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f82dc-d41f-4aaa-aaf2-becb170bdf5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-b72f82dc-d41f-4aaa-aaf2-becb170bdf5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:12:58.663927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:58.665441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:12:58.666602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:58.676900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 327/652] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:58.678916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f82dc-d41f-4aaa-aaf2-becb170bdf5c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:58.723220 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:58.726706 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e1d04-de4b-47a4-b4b4-cd26a21b4187 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:58.775075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d6743-d702-4ba4-a6b9-75cdb5ba0ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:58.775754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d6743-d702-4ba4-a6b9-75cdb5ba0ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:58.776040 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d6743-d702-4ba4-a6b9-75cdb5ba0ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:58.776316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d6743-d702-4ba4-a6b9-75cdb5ba0ac3 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:58.777578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d6743-d702-4ba4-a6b9-75cdb5ba0ac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:58.777886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d6743-d702-4ba4-a6b9-75cdb5ba0ac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:58.779299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d6743-d702-4ba4-a6b9-75cdb5ba0ac3 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:58.799572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 326/653] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:58.799572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d6743-d702-4ba4-a6b9-75cdb5ba0ac3 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:58.835323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f82dc-d41f-4aaa-aaf2-becb170bdf5c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:58.877491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297a0c4-ad56-48f2-bd96-137c3ed8e07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:58.878191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297a0c4-ad56-48f2-bd96-137c3ed8e07c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:58.878475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297a0c4-ad56-48f2-bd96-137c3ed8e07c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:58.878890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297a0c4-ad56-48f2-bd96-137c3ed8e07c None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:58.879747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297a0c4-ad56-48f2-bd96-137c3ed8e07c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:58.880064 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297a0c4-ad56-48f2-bd96-137c3ed8e07c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:58.881509 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297a0c4-ad56-48f2-bd96-137c3ed8e07c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:58.895742 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 328/654] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:58.896658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297a0c4-ad56-48f2-bd96-137c3ed8e07c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:58.908612 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:58.950542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d6743-d702-4ba4-a6b9-75cdb5ba0ac3 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:58.988861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f2ac6-1bdd-43c9-9673-6a6ea90acef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:58.989534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f2ac6-1bdd-43c9-9673-6a6ea90acef0 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:58.989968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f2ac6-1bdd-43c9-9673-6a6ea90acef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:58.990252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f2ac6-1bdd-43c9-9673-6a6ea90acef0 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:58.991101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f2ac6-1bdd-43c9-9673-6a6ea90acef0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:58.991513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f2ac6-1bdd-43c9-9673-6a6ea90acef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:58.992932 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f2ac6-1bdd-43c9-9673-6a6ea90acef0 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:58.998370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:59.004918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 327/655] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:12:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:59.006191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f2ac6-1bdd-43c9-9673-6a6ea90acef0 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:59.016823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297a0c4-ad56-48f2-bd96-137c3ed8e07c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:59.068625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763f051-f4cc-473a-b7e3-3833d7f790ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:59.070074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763f051-f4cc-473a-b7e3-3833d7f790ac None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:59.070657 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763f051-f4cc-473a-b7e3-3833d7f790ac None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:59.071192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763f051-f4cc-473a-b7e3-3833d7f790ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:59.082562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763f051-f4cc-473a-b7e3-3833d7f790ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:59.084600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763f051-f4cc-473a-b7e3-3833d7f790ac None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a763f051-f4cc-473a-b7e3-3833d7f790ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:12:59.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:59.104159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:59.105460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:12:59.105460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:12:59.105460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:12:59.105460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:12:59.105460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:12:59.105460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:12:59.105460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:12:59.105460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:59.105460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 329/656] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:12:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:59.105460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763f051-f4cc-473a-b7e3-3833d7f790ac None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:59.131074 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f2ac6-1bdd-43c9-9673-6a6ea90acef0 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:59.165582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:59.173885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242e8c0-66be-482d-89b7-03af32dfde78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:59.175047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242e8c0-66be-482d-89b7-03af32dfde78 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:59.175527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242e8c0-66be-482d-89b7-03af32dfde78 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:59.175991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242e8c0-66be-482d-89b7-03af32dfde78 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:59.186314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242e8c0-66be-482d-89b7-03af32dfde78 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:59.188336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242e8c0-66be-482d-89b7-03af32dfde78 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-2242e8c0-66be-482d-89b7-03af32dfde78 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:59.204826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:59.206028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:59.207097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:59.207097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:59.207097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:59.207097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:59.207097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:59.207097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:59.207097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:59.207097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:59.207097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 15:12:59.207097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:12:59.214883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 328/657] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:12:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:59.216153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242e8c0-66be-482d-89b7-03af32dfde78 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:59.238127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763f051-f4cc-473a-b7e3-3833d7f790ac None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:59.282250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4afc2-b4fa-402f-be77-14c5bea261e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:59.283072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4afc2-b4fa-402f-be77-14c5bea261e9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:59.283416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4afc2-b4fa-402f-be77-14c5bea261e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:59.283734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4afc2-b4fa-402f-be77-14c5bea261e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:59.302613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4afc2-b4fa-402f-be77-14c5bea261e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:59.304624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4afc2-b4fa-402f-be77-14c5bea261e9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f2e4afc2-b4fa-402f-be77-14c5bea261e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:12:59.330814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:59.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:59.332901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:12:59.332901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:12:59.332901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:12:59.332901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:59.332901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 330/658] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:59.333210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4afc2-b4fa-402f-be77-14c5bea261e9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:59.335535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242e8c0-66be-482d-89b7-03af32dfde78 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:59.371327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:59.375220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524c7b8-7116-4f7e-bf33-f8795066dc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:59.376541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524c7b8-7116-4f7e-bf33-f8795066dc33 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:59.377135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524c7b8-7116-4f7e-bf33-f8795066dc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:59.377618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f524c7b8-7116-4f7e-bf33-f8795066dc33 None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:59.378717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524c7b8-7116-4f7e-bf33-f8795066dc33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:59.379268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524c7b8-7116-4f7e-bf33-f8795066dc33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:59.381372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f524c7b8-7116-4f7e-bf33-f8795066dc33 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:59.402650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 329/659] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:12:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:59.404645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524c7b8-7116-4f7e-bf33-f8795066dc33 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:59.470476 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:59.511798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4afc2-b4fa-402f-be77-14c5bea261e9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:59.540462 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f524c7b8-7116-4f7e-bf33-f8795066dc33 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:59.553956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df97ff-df06-45cf-aa84-e4b9ea958940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:59.554767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df97ff-df06-45cf-aa84-e4b9ea958940 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:59.555124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df97ff-df06-45cf-aa84-e4b9ea958940 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:59.555470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df97ff-df06-45cf-aa84-e4b9ea958940 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:59.556653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df97ff-df06-45cf-aa84-e4b9ea958940 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:59.556828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df97ff-df06-45cf-aa84-e4b9ea958940 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:59.559260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df97ff-df06-45cf-aa84-e4b9ea958940 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:59.586698 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f761fa-cc07-457f-9436-aef1312dffbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:59.587338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 331/660] 10.222.0.93 () {56 vars in 1137 bytes} [Fri Jun 3 15:12:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:59.588391 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df97ff-df06-45cf-aa84-e4b9ea958940 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:59.592045 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f761fa-cc07-457f-9436-aef1312dffbc None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:59.592329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f761fa-cc07-457f-9436-aef1312dffbc None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:59.592687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f761fa-cc07-457f-9436-aef1312dffbc None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:59.593449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f761fa-cc07-457f-9436-aef1312dffbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:59.593764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f761fa-cc07-457f-9436-aef1312dffbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:59.595444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f761fa-cc07-457f-9436-aef1312dffbc None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:59.607968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 330/661] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:59.610558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ed22-8000-40ed-9715-956245ac813d None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:59.611080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ed22-8000-40ed-9715-956245ac813d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:59.611447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009ed22-8000-40ed-9715-956245ac813d None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:59.613805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 331/662] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:12:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:12:59.614661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009ed22-8000-40ed-9715-956245ac813d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:59.655233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:59.719312 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df97ff-df06-45cf-aa84-e4b9ea958940 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:59.736401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009ed22-8000-40ed-9715-956245ac813d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:59.747742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:59.765554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bda9de-8c1a-4142-99c6-d7b2c68df754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:59.766216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bda9de-8c1a-4142-99c6-d7b2c68df754 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:59.766573 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bda9de-8c1a-4142-99c6-d7b2c68df754 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:59.766802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bda9de-8c1a-4142-99c6-d7b2c68df754 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:59.767639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bda9de-8c1a-4142-99c6-d7b2c68df754 None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=c1966524091242e7a6cd2758027a0d4f) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:12:59.767950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bda9de-8c1a-4142-99c6-d7b2c68df754 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:59.769470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bda9de-8c1a-4142-99c6-d7b2c68df754 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:59.786563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 332/663] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:12:59 2022] GET /identity/v3/role_assignments?user.id=c1966524091242e7a6cd2758027a0d4f&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:12:59.786563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bda9de-8c1a-4142-99c6-d7b2c68df754 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:12:59.790709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f5c41-ce48-4018-bf8e-0ccf432eb979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:59.791498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f5c41-ce48-4018-bf8e-0ccf432eb979 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:59.791914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f5c41-ce48-4018-bf8e-0ccf432eb979 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:59.792811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f5c41-ce48-4018-bf8e-0ccf432eb979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:59.898832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f5c41-ce48-4018-bf8e-0ccf432eb979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:59.900999 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f5c41-ce48-4018-bf8e-0ccf432eb979 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:59.918783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bda9de-8c1a-4142-99c6-d7b2c68df754 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:12:59.939641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:12:59.960985 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de36b7da-572e-455e-9b35-d4756880275d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:12:59.962369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36b7da-572e-455e-9b35-d4756880275d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:59.962684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36b7da-572e-455e-9b35-d4756880275d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:59.962962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36b7da-572e-455e-9b35-d4756880275d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:12:59.972362 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36b7da-572e-455e-9b35-d4756880275d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:12:59.973860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de36b7da-572e-455e-9b35-d4756880275d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-de36b7da-572e-455e-9b35-d4756880275d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:12:59.989953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:12:59.991031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:12:59.991996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:12:59.991996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:12:59.991996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:12:59.991996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:12:59.991996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:12:59.991996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:12:59.991996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:12:59.991996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:12:59.991996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 03 15:12:59.991996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:12:59.991996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 333/664] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:12:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:12:59.993148 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d290-5d1a-49b6-809c-7240968d4a3f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:12:59.993148 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d290-5d1a-49b6-809c-7240968d4a3f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:12:59.993394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d290-5d1a-49b6-809c-7240968d4a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:00.012240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-74b4d290-5d1a-49b6-809c-7240968d4a3f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:00.013442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-74b4d290-5d1a-49b6-809c-7240968d4a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:00.055171 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 332/665] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:12:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:13:00.056113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f5c41-ce48-4018-bf8e-0ccf432eb979 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:00.074592 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:00.110582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 334/666] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:12:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:13:00.110582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4d290-5d1a-49b6-809c-7240968d4a3f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:00.204691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f5c41-ce48-4018-bf8e-0ccf432eb979 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:00.240892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4d290-5d1a-49b6-809c-7240968d4a3f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:00.244652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237de14c-1897-42ea-82ee-10291597a4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:00.245524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237de14c-1897-42ea-82ee-10291597a4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:00.245813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237de14c-1897-42ea-82ee-10291597a4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:00.246140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237de14c-1897-42ea-82ee-10291597a4f1 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:00.247028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237de14c-1897-42ea-82ee-10291597a4f1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:00.247339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237de14c-1897-42ea-82ee-10291597a4f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:00.248809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237de14c-1897-42ea-82ee-10291597a4f1 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:00.252555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:00.269535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 333/667] 10.222.0.93 () {56 vars in 1139 bytes} [Fri Jun 3 15:13:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:00.270417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237de14c-1897-42ea-82ee-10291597a4f1 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:00.288865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ace2a-a7cf-45c1-8e81-23bb05338f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:00.289974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ace2a-a7cf-45c1-8e81-23bb05338f52 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:00.290268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ace2a-a7cf-45c1-8e81-23bb05338f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:00.290570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ace2a-a7cf-45c1-8e81-23bb05338f52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:00.307702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ace2a-a7cf-45c1-8e81-23bb05338f52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:00.314330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ace2a-a7cf-45c1-8e81-23bb05338f52 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-3a9ace2a-a7cf-45c1-8e81-23bb05338f52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:00.340793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:00.341788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:00.343345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:00.343345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:00.343345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:00.343345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:00.346769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 335/668] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:00.348518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ace2a-a7cf-45c1-8e81-23bb05338f52 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:00.372160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:00.420831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237de14c-1897-42ea-82ee-10291597a4f1 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:00.463805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f5dd0-3c5a-4e1f-9157-bc9e1c96c168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:00.464469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f5dd0-3c5a-4e1f-9157-bc9e1c96c168 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:00.464750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f5dd0-3c5a-4e1f-9157-bc9e1c96c168 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:00.465026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f5dd0-3c5a-4e1f-9157-bc9e1c96c168 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:00.477312 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f5dd0-3c5a-4e1f-9157-bc9e1c96c168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:00.477770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ace2a-a7cf-45c1-8e81-23bb05338f52 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-d82f5dd0-3c5a-4e1f-9157-bc9e1c96c168 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:00.486039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:00.487092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 334/669] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:13:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:00.488560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f5dd0-3c5a-4e1f-9157-bc9e1c96c168 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:00.531012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e8cd2-c066-4425-9946-2844496b739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:00.531720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e8cd2-c066-4425-9946-2844496b739d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:00.532003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e8cd2-c066-4425-9946-2844496b739d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:00.532333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e8cd2-c066-4425-9946-2844496b739d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:00.563733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:00.612625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e8cd2-c066-4425-9946-2844496b739d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:00.614150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e8cd2-c066-4425-9946-2844496b739d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:00.624568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f5dd0-3c5a-4e1f-9157-bc9e1c96c168 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:00.657267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:00.667647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee697c-c7ad-431d-9edc-a2a6b9472321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:00.668381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee697c-c7ad-431d-9edc-a2a6b9472321 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:00.668738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee697c-c7ad-431d-9edc-a2a6b9472321 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:00.669024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee697c-c7ad-431d-9edc-a2a6b9472321 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:00.669761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee697c-c7ad-431d-9edc-a2a6b9472321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:00.670064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee697c-c7ad-431d-9edc-a2a6b9472321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:00.673768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee697c-c7ad-431d-9edc-a2a6b9472321 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:00.684132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 335/670] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:00.684961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee697c-c7ad-431d-9edc-a2a6b9472321 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:00.738668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 336/671] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:13:00.739737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63352fc7-c357-4ddd-a0dc-2eb3087564fe None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:00.740024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63352fc7-c357-4ddd-a0dc-2eb3087564fe None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:00.740298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63352fc7-c357-4ddd-a0dc-2eb3087564fe None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:00.745868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 337/672] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:00.746693 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63352fc7-c357-4ddd-a0dc-2eb3087564fe None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:00.814456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:00.815728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee697c-c7ad-431d-9edc-a2a6b9472321 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:00.854161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa5a88-7c9c-4dfe-a78b-057cdb3d2cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:00.854161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa5a88-7c9c-4dfe-a78b-057cdb3d2cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:00.854161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa5a88-7c9c-4dfe-a78b-057cdb3d2cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:00.855313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa5a88-7c9c-4dfe-a78b-057cdb3d2cd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:00.856357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa5a88-7c9c-4dfe-a78b-057cdb3d2cd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:00.856671 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa5a88-7c9c-4dfe-a78b-057cdb3d2cd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:00.858143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa5a88-7c9c-4dfe-a78b-057cdb3d2cd6 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:00.872279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63352fc7-c357-4ddd-a0dc-2eb3087564fe None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:00.872943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 336/673] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:13:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:00.873892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5b985-5a14-4c47-8ed9-a7787f856505 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:00.874183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5b985-5a14-4c47-8ed9-a7787f856505 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:00.874458 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5b985-5a14-4c47-8ed9-a7787f856505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:00.891183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-dac5b985-5a14-4c47-8ed9-a7787f856505 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:00.892461 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-dac5b985-5a14-4c47-8ed9-a7787f856505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:00.917377 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9f8df-2588-45dd-9278-e9aaf6de2a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:00.918167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9f8df-2588-45dd-9278-e9aaf6de2a77 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:00.918451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9f8df-2588-45dd-9278-e9aaf6de2a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:00.918782 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9f8df-2588-45dd-9278-e9aaf6de2a77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:00.925678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9f8df-2588-45dd-9278-e9aaf6de2a77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:00.927266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9f8df-2588-45dd-9278-e9aaf6de2a77 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:00.942202 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7ed9f8df-2588-45dd-9278-e9aaf6de2a77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:00.943269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:00.944244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:00.945156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:00.945156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:00.945156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:00.945156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:00.945156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:00.945156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:00.945156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:00.945156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:00.949609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 338/674] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:00.950514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9f8df-2588-45dd-9278-e9aaf6de2a77 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:00.970610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 337/675] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:13:00.970610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5b985-5a14-4c47-8ed9-a7787f856505 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:01.066285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9f8df-2588-45dd-9278-e9aaf6de2a77 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:01.087767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:01.100684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5b985-5a14-4c47-8ed9-a7787f856505 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:01.105151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce515242-6d43-4995-99f2-25117478ff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:01.105963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce515242-6d43-4995-99f2-25117478ff88 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:01.106245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce515242-6d43-4995-99f2-25117478ff88 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:01.106522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce515242-6d43-4995-99f2-25117478ff88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:01.119282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce515242-6d43-4995-99f2-25117478ff88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:01.121614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce515242-6d43-4995-99f2-25117478ff88 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:01.139344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9ca9f-b753-4c12-a2ff-b81993f6c31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:01.140141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ca9f-b753-4c12-a2ff-b81993f6c31d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:01.140421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ca9f-b753-4c12-a2ff-b81993f6c31d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:01.140695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ca9f-b753-4c12-a2ff-b81993f6c31d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ce515242-6d43-4995-99f2-25117478ff88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:01.141629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:01.142495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:01.143856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:01.143856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:01.143856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:01.143856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:01.143856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 339/676] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:01.144341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce515242-6d43-4995-99f2-25117478ff88 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:01.155734 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9ca9f-b753-4c12-a2ff-b81993f6c31d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:01.157199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9ca9f-b753-4c12-a2ff-b81993f6c31d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-aee9ca9f-b753-4c12-a2ff-b81993f6c31d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:01.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:01.176883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:01.177756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:01.177756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:01.177756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:01.177756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:01.177756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 338/677] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:01.178036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9ca9f-b753-4c12-a2ff-b81993f6c31d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:01.229148 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:01.273029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce515242-6d43-4995-99f2-25117478ff88 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:01.311923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd6129-ee79-43a6-a4ab-bfbcdd781c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:01.313361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd6129-ee79-43a6-a4ab-bfbcdd781c82 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:01.313707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9ca9f-b753-4c12-a2ff-b81993f6c31d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:01.314026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd6129-ee79-43a6-a4ab-bfbcdd781c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:01.314472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd6129-ee79-43a6-a4ab-bfbcdd781c82 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:01.315572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd6129-ee79-43a6-a4ab-bfbcdd781c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:01.316163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd6129-ee79-43a6-a4ab-bfbcdd781c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:01.317848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd6129-ee79-43a6-a4ab-bfbcdd781c82 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:01.336903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 340/678] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:01.336903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd6129-ee79-43a6-a4ab-bfbcdd781c82 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:01.357881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3631571-1d68-4a72-a57c-19d00b11b31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:01.358728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3631571-1d68-4a72-a57c-19d00b11b31c None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:01.359010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3631571-1d68-4a72-a57c-19d00b11b31c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:01.359277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3631571-1d68-4a72-a57c-19d00b11b31c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:01.371537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3631571-1d68-4a72-a57c-19d00b11b31c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:01.372088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:01.372857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3631571-1d68-4a72-a57c-19d00b11b31c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-a3631571-1d68-4a72-a57c-19d00b11b31c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:01.391317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:01.392253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:01.393163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:01.393163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:01.393163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:01.393163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:01.393163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 339/679] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:01.393447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3631571-1d68-4a72-a57c-19d00b11b31c None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:01.464340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd6129-ee79-43a6-a4ab-bfbcdd781c82 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:01.503197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd046513-8e75-4a2c-9692-70dccab09bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:01.504255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd046513-8e75-4a2c-9692-70dccab09bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:01.504728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd046513-8e75-4a2c-9692-70dccab09bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:01.505204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd046513-8e75-4a2c-9692-70dccab09bf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:01.506124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd046513-8e75-4a2c-9692-70dccab09bf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:01.506653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd046513-8e75-4a2c-9692-70dccab09bf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:01.508225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd046513-8e75-4a2c-9692-70dccab09bf8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:01.520572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 341/680] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:01.522010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:01.525687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd046513-8e75-4a2c-9692-70dccab09bf8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:01.544571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3631571-1d68-4a72-a57c-19d00b11b31c None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:01.586059 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf2fa7-5dc6-49f9-b022-2ae5489023b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:01.586870 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf2fa7-5dc6-49f9-b022-2ae5489023b1 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:01.587337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf2fa7-5dc6-49f9-b022-2ae5489023b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:01.587616 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf2fa7-5dc6-49f9-b022-2ae5489023b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:01.588738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf2fa7-5dc6-49f9-b022-2ae5489023b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:01.589634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf2fa7-5dc6-49f9-b022-2ae5489023b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:01.591414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf2fa7-5dc6-49f9-b022-2ae5489023b1 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:01.606294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 340/681] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:01.607065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf2fa7-5dc6-49f9-b022-2ae5489023b1 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:01.641832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:01.657532 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd046513-8e75-4a2c-9692-70dccab09bf8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:01.701591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbe4e5-ef72-4c8d-8fff-499250a668f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:01.702427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbe4e5-ef72-4c8d-8fff-499250a668f6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:01.702797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbe4e5-ef72-4c8d-8fff-499250a668f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:01.703087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbe4e5-ef72-4c8d-8fff-499250a668f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:01.703931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbe4e5-ef72-4c8d-8fff-499250a668f6 None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=98aa51c4bc604b0ca8a6c69bc1455bbb) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:01.704248 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbe4e5-ef72-4c8d-8fff-499250a668f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:01.705699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbe4e5-ef72-4c8d-8fff-499250a668f6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:01.718576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 342/682] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:01 2022] GET /identity/v3/role_assignments?user.id=98aa51c4bc604b0ca8a6c69bc1455bbb&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:01.718576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbe4e5-ef72-4c8d-8fff-499250a668f6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:01.740281 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf2fa7-5dc6-49f9-b022-2ae5489023b1 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:01.797323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940efc47-c9bc-4292-91e6-4eda80f23299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:01.798808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940efc47-c9bc-4292-91e6-4eda80f23299 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:01.799521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940efc47-c9bc-4292-91e6-4eda80f23299 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:01.800223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940efc47-c9bc-4292-91e6-4eda80f23299 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:01.801930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940efc47-c9bc-4292-91e6-4eda80f23299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:01.803571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940efc47-c9bc-4292-91e6-4eda80f23299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:01.805800 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940efc47-c9bc-4292-91e6-4eda80f23299 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:01.836738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 341/683] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:01.838150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940efc47-c9bc-4292-91e6-4eda80f23299 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:01.844857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:01.877284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbe4e5-ef72-4c8d-8fff-499250a668f6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:01.956210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc217-ff1f-458d-8816-35169ecbfa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:01.957450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc217-ff1f-458d-8816-35169ecbfa9c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:01.957835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc217-ff1f-458d-8816-35169ecbfa9c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:01.958260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc217-ff1f-458d-8816-35169ecbfa9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:01.980796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edc217-ff1f-458d-8816-35169ecbfa9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:01.981642 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edc217-ff1f-458d-8816-35169ecbfa9c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:02.000034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940efc47-c9bc-4292-91e6-4eda80f23299 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-46edc217-ff1f-458d-8816-35169ecbfa9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:02.012496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:02.014678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:02.015601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:02.015601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:02.015601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:02.015601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:02.015601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:02.015601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:02.015601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:02.015601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:02.015601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 343/684] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:02.016638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc217-ff1f-458d-8816-35169ecbfa9c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:02.049396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:02.053823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1fb15-4b8f-4948-8f4b-93a2f896d605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:02.055506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1fb15-4b8f-4948-8f4b-93a2f896d605 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:02.056148 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1fb15-4b8f-4948-8f4b-93a2f896d605 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:02.057007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1fb15-4b8f-4948-8f4b-93a2f896d605 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:02.072859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1fb15-4b8f-4948-8f4b-93a2f896d605 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:02.075229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1fb15-4b8f-4948-8f4b-93a2f896d605 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-31b1fb15-4b8f-4948-8f4b-93a2f896d605 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:02.092498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:02.093682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:02.094892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:02.094892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:02.094892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:02.094892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:02.094892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:02.094892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:02.094892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:02.094892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:02.094892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:13:02.094892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:02.096890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 342/685] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:02.102096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1fb15-4b8f-4948-8f4b-93a2f896d605 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:02.160954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc217-ff1f-458d-8816-35169ecbfa9c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:02.172911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:02.208083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107808aa-e8f8-4a90-bd81-092e9e382dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:02.208760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107808aa-e8f8-4a90-bd81-092e9e382dd0 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:02.209044 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107808aa-e8f8-4a90-bd81-092e9e382dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:02.209316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107808aa-e8f8-4a90-bd81-092e9e382dd0 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:02.209944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1fb15-4b8f-4948-8f4b-93a2f896d605 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:02.210069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107808aa-e8f8-4a90-bd81-092e9e382dd0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:02.211473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107808aa-e8f8-4a90-bd81-092e9e382dd0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:02.238009 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 344/686] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:13:02 2022] POST /identity/v3/users => generated 254 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:02.239040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107808aa-e8f8-4a90-bd81-092e9e382dd0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:02.272008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96275401-ba1b-4027-89e4-99a7a8fae40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:02.272938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96275401-ba1b-4027-89e4-99a7a8fae40b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:02.273413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96275401-ba1b-4027-89e4-99a7a8fae40b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:02.273788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96275401-ba1b-4027-89e4-99a7a8fae40b None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:02.274786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96275401-ba1b-4027-89e4-99a7a8fae40b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:02.275218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96275401-ba1b-4027-89e4-99a7a8fae40b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:02.277527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96275401-ba1b-4027-89e4-99a7a8fae40b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:02.309911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 343/687] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:02.323166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96275401-ba1b-4027-89e4-99a7a8fae40b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:02.330927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:02.377539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107808aa-e8f8-4a90-bd81-092e9e382dd0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:02.427567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beb3b8-8211-48bf-8a1b-2144cea5c3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:02.428288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beb3b8-8211-48bf-8a1b-2144cea5c3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:02.428569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beb3b8-8211-48bf-8a1b-2144cea5c3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:02.428840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beb3b8-8211-48bf-8a1b-2144cea5c3a5 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:02.430238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beb3b8-8211-48bf-8a1b-2144cea5c3a5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:02.430582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beb3b8-8211-48bf-8a1b-2144cea5c3a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:02.433414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beb3b8-8211-48bf-8a1b-2144cea5c3a5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:02.458309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96275401-ba1b-4027-89e4-99a7a8fae40b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:02.458948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 345/688] 10.222.0.93 () {56 vars in 1135 bytes} [Fri Jun 3 15:13:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:02.460716 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2cf4-b337-4829-804a-76d22b06f9f4 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:02.461010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2cf4-b337-4829-804a-76d22b06f9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:02.461292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd2cf4-b337-4829-804a-76d22b06f9f4 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:02.463360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 346/689] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:02.468588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd2cf4-b337-4829-804a-76d22b06f9f4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:02.500551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:02.516231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c72c10-d6bb-4197-8b0f-31c9cfac9a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:02.517362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c72c10-d6bb-4197-8b0f-31c9cfac9a7a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:02.517890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c72c10-d6bb-4197-8b0f-31c9cfac9a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:02.518275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c72c10-d6bb-4197-8b0f-31c9cfac9a7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:02.519340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c72c10-d6bb-4197-8b0f-31c9cfac9a7a None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=e3f887d5bbcd424884537b95affd96f8) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:02.520268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c72c10-d6bb-4197-8b0f-31c9cfac9a7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:02.522462 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c72c10-d6bb-4197-8b0f-31c9cfac9a7a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:02.535918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 344/690] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:02 2022] GET /identity/v3/role_assignments?user.id=e3f887d5bbcd424884537b95affd96f8&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:02.539121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4647d-11ee-4b28-8f43-96896fb26432 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:02.540217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4647d-11ee-4b28-8f43-96896fb26432 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:02.541280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4647d-11ee-4b28-8f43-96896fb26432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:02.572374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-74f4647d-11ee-4b28-8f43-96896fb26432 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:02.575730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-74f4647d-11ee-4b28-8f43-96896fb26432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:02.637002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd2cf4-b337-4829-804a-76d22b06f9f4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:02.675900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:02.685069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb098228-7b9e-44f2-ba9d-93518105ac42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:02.685907 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb098228-7b9e-44f2-ba9d-93518105ac42 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:02.686327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb098228-7b9e-44f2-ba9d-93518105ac42 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:02.686698 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb098228-7b9e-44f2-ba9d-93518105ac42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:02.689236 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 345/691] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:13:02.720684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4647d-11ee-4b28-8f43-96896fb26432 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:02.768095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb098228-7b9e-44f2-ba9d-93518105ac42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:02.769858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb098228-7b9e-44f2-ba9d-93518105ac42 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:02.875742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4647d-11ee-4b28-8f43-96896fb26432 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:02.884287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 347/692] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:13:02.885625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb098228-7b9e-44f2-ba9d-93518105ac42 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:02.897519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:02.918395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:02.922069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ca3c5-d0bd-48cf-9c35-0488be852970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:02.923691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca3c5-d0bd-48cf-9c35-0488be852970 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:02.924119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca3c5-d0bd-48cf-9c35-0488be852970 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:02.924722 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ca3c5-d0bd-48cf-9c35-0488be852970 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:02.937647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ca3c5-d0bd-48cf-9c35-0488be852970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-420ca3c5-d0bd-48cf-9c35-0488be852970 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:02.946065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:02.947339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:02.948505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:02.951200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 346/693] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:02.959669 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ca3c5-d0bd-48cf-9c35-0488be852970 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:03.001614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb098228-7b9e-44f2-ba9d-93518105ac42 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:03.037992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152bb1d-70d8-47d3-9308-b0c5231db271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:03.039072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152bb1d-70d8-47d3-9308-b0c5231db271 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:03.039358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152bb1d-70d8-47d3-9308-b0c5231db271 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:03.039635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152bb1d-70d8-47d3-9308-b0c5231db271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:03.051670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152bb1d-70d8-47d3-9308-b0c5231db271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:03.053295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152bb1d-70d8-47d3-9308-b0c5231db271 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a152bb1d-70d8-47d3-9308-b0c5231db271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:03.074484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:03.075983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:03.077493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:03.077493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:03.077493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:03.077493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:03.077493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 348/694] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:03.081702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152bb1d-70d8-47d3-9308-b0c5231db271 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:03.131668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ca3c5-d0bd-48cf-9c35-0488be852970 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:03.197029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a9df6-9ec2-4a1a-aff8-807cfaab9f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:03.198693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a9df6-9ec2-4a1a-aff8-807cfaab9f94 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:03.199212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a9df6-9ec2-4a1a-aff8-807cfaab9f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:03.200261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a9df6-9ec2-4a1a-aff8-807cfaab9f94 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:03.201441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a9df6-9ec2-4a1a-aff8-807cfaab9f94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:03.201966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a9df6-9ec2-4a1a-aff8-807cfaab9f94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:03.203770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a9df6-9ec2-4a1a-aff8-807cfaab9f94 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:03.215025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:03.222852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152bb1d-70d8-47d3-9308-b0c5231db271 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:03.235419 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 347/695] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:13:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:03.241063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a9df6-9ec2-4a1a-aff8-807cfaab9f94 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:03.282391 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689d07c-2887-462c-8670-1c21a3c2a99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:03.284050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689d07c-2887-462c-8670-1c21a3c2a99b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:03.284335 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689d07c-2887-462c-8670-1c21a3c2a99b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:03.285380 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689d07c-2887-462c-8670-1c21a3c2a99b None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:03.286126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689d07c-2887-462c-8670-1c21a3c2a99b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:03.286432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689d07c-2887-462c-8670-1c21a3c2a99b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:03.287864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1689d07c-2887-462c-8670-1c21a3c2a99b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:03.313649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:03.316301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 349/696] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:03.323544 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689d07c-2887-462c-8670-1c21a3c2a99b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:03.429380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a9df6-9ec2-4a1a-aff8-807cfaab9f94 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:03.466173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1689d07c-2887-462c-8670-1c21a3c2a99b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:03.468384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bfa3a-8253-4bd5-8cdb-de154e59f7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:03.469272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bfa3a-8253-4bd5-8cdb-de154e59f7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:03.469560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bfa3a-8253-4bd5-8cdb-de154e59f7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:03.469832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bfa3a-8253-4bd5-8cdb-de154e59f7f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:03.482493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784bfa3a-8253-4bd5-8cdb-de154e59f7f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:03.484111 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784bfa3a-8253-4bd5-8cdb-de154e59f7f7 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-784bfa3a-8253-4bd5-8cdb-de154e59f7f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:03.503586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:03.506483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:03.509212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:03.509212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:03.509212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:03.509212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:03.509660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7029f5-bcd2-4de8-865c-dac17b0e4b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:03.509660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7029f5-bcd2-4de8-865c-dac17b0e4b33 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:03.509660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7029f5-bcd2-4de8-865c-dac17b0e4b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:03.509660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7029f5-bcd2-4de8-865c-dac17b0e4b33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:03.510215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 348/697] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:03.514559 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bfa3a-8253-4bd5-8cdb-de154e59f7f7 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:03.514829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7029f5-bcd2-4de8-865c-dac17b0e4b33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:03.515015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7029f5-bcd2-4de8-865c-dac17b0e4b33 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9a7029f5-bcd2-4de8-865c-dac17b0e4b33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:03.530627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:03.532765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:03.535580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:03.535580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:03.535580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:03.535580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:03.535580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:03.535580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:03.535580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:03.535580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:03.537326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 350/698] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:03.538634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7029f5-bcd2-4de8-865c-dac17b0e4b33 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:03.550728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:03.664436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:03.676322 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784bfa3a-8253-4bd5-8cdb-de154e59f7f7 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:03.722371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c98a4-6826-438b-8951-df4d4105683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:03.722983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c98a4-6826-438b-8951-df4d4105683e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:03.723265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c98a4-6826-438b-8951-df4d4105683e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:03.723541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c98a4-6826-438b-8951-df4d4105683e None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:03.724268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c98a4-6826-438b-8951-df4d4105683e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:03.724574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c98a4-6826-438b-8951-df4d4105683e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:03.725955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c98a4-6826-438b-8951-df4d4105683e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:03.740566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 349/699] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:03.751587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c98a4-6826-438b-8951-df4d4105683e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:03.781982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7029f5-bcd2-4de8-865c-dac17b0e4b33 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:03.823210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c38603-d971-4bdb-a3e9-687fc4984f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:03.824502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c38603-d971-4bdb-a3e9-687fc4984f67 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:03.825021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c38603-d971-4bdb-a3e9-687fc4984f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:03.825529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c38603-d971-4bdb-a3e9-687fc4984f67 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:03.826523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c38603-d971-4bdb-a3e9-687fc4984f67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:03.827062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c38603-d971-4bdb-a3e9-687fc4984f67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:03.828832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c38603-d971-4bdb-a3e9-687fc4984f67 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:03.842730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 351/700] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:03.853213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c38603-d971-4bdb-a3e9-687fc4984f67 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:03.910327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c98a4-6826-438b-8951-df4d4105683e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:03.969802 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4b52b-de9e-4a53-9650-d34e95c69c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:03.970643 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4b52b-de9e-4a53-9650-d34e95c69c2b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:03.970930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4b52b-de9e-4a53-9650-d34e95c69c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:03.971204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4b52b-de9e-4a53-9650-d34e95c69c2b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:03.986778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4b52b-de9e-4a53-9650-d34e95c69c2b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:03.988181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4b52b-de9e-4a53-9650-d34e95c69c2b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:03.998945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-3fb4b52b-de9e-4a53-9650-d34e95c69c2b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:04.022018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:04.023069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:04.024180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:04.024180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:04.024180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:04.024180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:04.024180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:04.024180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:04.024180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:04.024180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:04.024180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:13:04.024180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:04.024180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 350/701] 10.222.0.93 () {56 vars in 1093 bytes} [Fri Jun 3 15:13:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:04.033347 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4b52b-de9e-4a53-9650-d34e95c69c2b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:04.064298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:04.077688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c38603-d971-4bdb-a3e9-687fc4984f67 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:04.155650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf55942-4ab2-4cd4-95d1-092e02157d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:04.156352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55942-4ab2-4cd4-95d1-092e02157d6c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:04.156635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55942-4ab2-4cd4-95d1-092e02157d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:04.156910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf55942-4ab2-4cd4-95d1-092e02157d6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:04.157768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf55942-4ab2-4cd4-95d1-092e02157d6c None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=bb4fb8f859dc4489b8c4fd8fdc09c3f6) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:04.158099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf55942-4ab2-4cd4-95d1-092e02157d6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:04.160211 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf55942-4ab2-4cd4-95d1-092e02157d6c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:04.176021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 352/702] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:03 2022] GET /identity/v3/role_assignments?user.id=bb4fb8f859dc4489b8c4fd8fdc09c3f6&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 608 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:04.229805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4b52b-de9e-4a53-9650-d34e95c69c2b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:04.293355 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111bd32-5258-43f3-8893-06882fdff954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:04.294047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111bd32-5258-43f3-8893-06882fdff954 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:04.294331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111bd32-5258-43f3-8893-06882fdff954 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:04.294638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111bd32-5258-43f3-8893-06882fdff954 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:04.295500 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111bd32-5258-43f3-8893-06882fdff954 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:04.295813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111bd32-5258-43f3-8893-06882fdff954 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:04.297319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111bd32-5258-43f3-8893-06882fdff954 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:04.333045 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 351/703] 10.222.0.93 () {56 vars in 1133 bytes} [Fri Jun 3 15:13:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:04.339499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf55942-4ab2-4cd4-95d1-092e02157d6c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:04.469148 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf55942-4ab2-4cd4-95d1-092e02157d6c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:04.496418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:04.512269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceec27-fe49-4da2-b4af-6ee5a5a856df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:04.512932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceec27-fe49-4da2-b4af-6ee5a5a856df None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:04.513216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceec27-fe49-4da2-b4af-6ee5a5a856df None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:04.513491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ceec27-fe49-4da2-b4af-6ee5a5a856df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:04.599566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceec27-fe49-4da2-b4af-6ee5a5a856df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:04.601005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ceec27-fe49-4da2-b4af-6ee5a5a856df None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:04.705901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b83e1-c176-444f-b2c5-fb8442912a7c None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:04.706217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b83e1-c176-444f-b2c5-fb8442912a7c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:04.706480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b83e1-c176-444f-b2c5-fb8442912a7c None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:04.708117 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 352/704] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:13:04.713850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdffb0a-a929-4012-b8d5-a0dcc67b09df None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:04.714421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdffb0a-a929-4012-b8d5-a0dcc67b09df None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:04.714934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdffb0a-a929-4012-b8d5-a0dcc67b09df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:04.716178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 353/705] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:13:04.722286 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceec27-fe49-4da2-b4af-6ee5a5a856df None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:04.735570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2cdffb0a-a929-4012-b8d5-a0dcc67b09df None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:04.737125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2cdffb0a-a929-4012-b8d5-a0dcc67b09df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:04.754991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:04.811380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:04.819945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 353/706] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 03 15:13:04.841483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ceec27-fe49-4da2-b4af-6ee5a5a856df None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:04.847194 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdffb0a-a929-4012-b8d5-a0dcc67b09df None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:04.880404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e8244-bcc6-4215-91f8-4a09cd0535ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:04.881125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e8244-bcc6-4215-91f8-4a09cd0535ed None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:04.881402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e8244-bcc6-4215-91f8-4a09cd0535ed None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:04.881691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e8244-bcc6-4215-91f8-4a09cd0535ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:04.895801 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e8244-bcc6-4215-91f8-4a09cd0535ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:04.897295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e8244-bcc6-4215-91f8-4a09cd0535ed None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-669e8244-bcc6-4215-91f8-4a09cd0535ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:04.917909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:04.919171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:04.920307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:04.920307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:04.920307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:04.920307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:04.920307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 354/707] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:04.925172 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e8244-bcc6-4215-91f8-4a09cd0535ed None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:04.973583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdffb0a-a929-4012-b8d5-a0dcc67b09df None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:05.006932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:05.014191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f2f6a-1272-4b50-a26d-df8de6cc5017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:05.015426 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f2f6a-1272-4b50-a26d-df8de6cc5017 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.015914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f2f6a-1272-4b50-a26d-df8de6cc5017 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.016398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f2f6a-1272-4b50-a26d-df8de6cc5017 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.017264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f2f6a-1272-4b50-a26d-df8de6cc5017 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:05.018971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f2f6a-1272-4b50-a26d-df8de6cc5017 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:05.030015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e8244-bcc6-4215-91f8-4a09cd0535ed None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-3e0f2f6a-1272-4b50-a26d-df8de6cc5017 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:05.034708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:05.035702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 15:13:05.036557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:05.044957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 354/708] 10.222.0.93 () {56 vars in 1129 bytes} [Fri Jun 3 15:13:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:05.046501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f2f6a-1272-4b50-a26d-df8de6cc5017 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:05.066751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f3635-acad-4d11-bf55-22cb81d4381a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:05.067438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f3635-acad-4d11-bf55-22cb81d4381a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.067693 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f3635-acad-4d11-bf55-22cb81d4381a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.067966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f3635-acad-4d11-bf55-22cb81d4381a None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.068692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f3635-acad-4d11-bf55-22cb81d4381a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:05.068993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f3635-acad-4d11-bf55-22cb81d4381a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:05.070789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f3635-acad-4d11-bf55-22cb81d4381a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:05.083813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 355/709] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:05.089191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f3635-acad-4d11-bf55-22cb81d4381a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:05.152046 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:05.197043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f3635-acad-4d11-bf55-22cb81d4381a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:05.246550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f2f6a-1272-4b50-a26d-df8de6cc5017 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:05.247825 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60930a3-9ecd-4f68-bd45-1de6102c1339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:05.248646 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60930a3-9ecd-4f68-bd45-1de6102c1339 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.248930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60930a3-9ecd-4f68-bd45-1de6102c1339 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.249205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60930a3-9ecd-4f68-bd45-1de6102c1339 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.256779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60930a3-9ecd-4f68-bd45-1de6102c1339 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:05.258261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60930a3-9ecd-4f68-bd45-1de6102c1339 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:05.262317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c60930a3-9ecd-4f68-bd45-1de6102c1339 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:05.275863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:05.277170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:05.278852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:05.278852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:05.278852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:05.278852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:05.278852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:05.278852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:05.278852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:05.278852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:05.278852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 356/710] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:05.284089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60930a3-9ecd-4f68-bd45-1de6102c1339 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:05.319435 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6965a-f429-4966-ae7a-0ad140de28b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:05.320166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6965a-f429-4966-ae7a-0ad140de28b5 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.320449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6965a-f429-4966-ae7a-0ad140de28b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.320721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6965a-f429-4966-ae7a-0ad140de28b5 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.321442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6965a-f429-4966-ae7a-0ad140de28b5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:05.321846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6965a-f429-4966-ae7a-0ad140de28b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:05.323601 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6965a-f429-4966-ae7a-0ad140de28b5 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:05.329496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 355/711] 10.222.0.93 () {56 vars in 1124 bytes} [Fri Jun 3 15:13:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:05.331452 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff6322-8df1-4eab-8224-6cba5a7ed7f6 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.331762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff6322-8df1-4eab-8224-6cba5a7ed7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.332037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff6322-8df1-4eab-8224-6cba5a7ed7f6 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.337655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 356/712] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:05.338748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff6322-8df1-4eab-8224-6cba5a7ed7f6 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:05.404000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60930a3-9ecd-4f68-bd45-1de6102c1339 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:05.440166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67325515-8a51-4cfb-a1e6-a2f4eac33008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:05.440834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325515-8a51-4cfb-a1e6-a2f4eac33008 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.441124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325515-8a51-4cfb-a1e6-a2f4eac33008 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.441368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67325515-8a51-4cfb-a1e6-a2f4eac33008 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.442165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67325515-8a51-4cfb-a1e6-a2f4eac33008 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:05.442471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67325515-8a51-4cfb-a1e6-a2f4eac33008 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:05.443958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67325515-8a51-4cfb-a1e6-a2f4eac33008 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:05.456873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 357/713] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:05.458152 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6eda9-3979-4552-ba60-6e568e7e43de None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.458439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6eda9-3979-4552-ba60-6e568e7e43de None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.458743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6eda9-3979-4552-ba60-6e568e7e43de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.471142 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff6322-8df1-4eab-8224-6cba5a7ed7f6 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:05.484241 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e9a6eda9-3979-4552-ba60-6e568e7e43de None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:05.485452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e9a6eda9-3979-4552-ba60-6e568e7e43de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:05.505945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:05.514948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375c024-9093-4e59-bd19-a44720dc6205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:05.516078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375c024-9093-4e59-bd19-a44720dc6205 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.516218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375c024-9093-4e59-bd19-a44720dc6205 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.516560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375c024-9093-4e59-bd19-a44720dc6205 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.517387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375c024-9093-4e59-bd19-a44720dc6205 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:05.517843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375c024-9093-4e59-bd19-a44720dc6205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:05.519356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375c024-9093-4e59-bd19-a44720dc6205 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:05.525611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 357/714] 10.222.0.93 () {56 vars in 1124 bytes} [Fri Jun 3 15:13:05 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:05.527131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375c024-9093-4e59-bd19-a44720dc6205 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:05.582726 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:05.630688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 358/715] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:13:05.684184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6eda9-3979-4552-ba60-6e568e7e43de None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:05.691871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375c024-9093-4e59-bd19-a44720dc6205 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:05.731558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772bf90-6996-40b1-8a03-8ec389a0d066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:05.732261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772bf90-6996-40b1-8a03-8ec389a0d066 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.732548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772bf90-6996-40b1-8a03-8ec389a0d066 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.732826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772bf90-6996-40b1-8a03-8ec389a0d066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.733906 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772bf90-6996-40b1-8a03-8ec389a0d066 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=23546323ca33412fb388afe308a97499, user.id=c1966524091242e7a6cd2758027a0d4f) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:05.734030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772bf90-6996-40b1-8a03-8ec389a0d066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:05.735517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772bf90-6996-40b1-8a03-8ec389a0d066 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:05.741624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 358/716] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:05 2022] GET /identity/v3/role_assignments?user.id=c1966524091242e7a6cd2758027a0d4f&scope.project.id=23546323ca33412fb388afe308a97499&role.id=d67707782358417ba9f25b2793ca2890 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:05.743044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa53b1-bfe2-4197-8e92-23cda9987602 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.743334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa53b1-bfe2-4197-8e92-23cda9987602 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.743616 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa53b1-bfe2-4197-8e92-23cda9987602 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.746619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 359/717] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:05.751795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ad8bc-483a-4320-aff8-46264c574ca7 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.752109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ad8bc-483a-4320-aff8-46264c574ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.752384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ad8bc-483a-4320-aff8-46264c574ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.771297 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-576ad8bc-483a-4320-aff8-46264c574ca7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:05.773340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-576ad8bc-483a-4320-aff8-46264c574ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:05.859066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6eda9-3979-4552-ba60-6e568e7e43de None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:05.874574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:05.915840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 360/718] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:13:05.927245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0c4d4-d0f0-4dec-8207-fc263233709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:05.927976 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0c4d4-d0f0-4dec-8207-fc263233709f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:05.928256 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0c4d4-d0f0-4dec-8207-fc263233709f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:05.928531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0c4d4-d0f0-4dec-8207-fc263233709f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:05.948101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0c4d4-d0f0-4dec-8207-fc263233709f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-79d0c4d4-d0f0-4dec-8207-fc263233709f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:05.956081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:05.958425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:05.960550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 359/719] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:13:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:05.966558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ad8bc-483a-4320-aff8-46264c574ca7 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:05.980036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0c4d4-d0f0-4dec-8207-fc263233709f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:06.052829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:06.116026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0c4d4-d0f0-4dec-8207-fc263233709f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:06.152750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ad8bc-483a-4320-aff8-46264c574ca7 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:06.155611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bb74c-f89c-42c7-b5ca-852a9e3f229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:06.156439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bb74c-f89c-42c7-b5ca-852a9e3f229a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:06.156724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bb74c-f89c-42c7-b5ca-852a9e3f229a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:06.157005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bb74c-f89c-42c7-b5ca-852a9e3f229a None admin] PATH_INFO: `/v3/services/network` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:06.157685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bb74c-f89c-42c7-b5ca-852a9e3f229a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:06.159167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bb74c-f89c-42c7-b5ca-852a9e3f229a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-246bb74c-f89c-42c7-b5ca-852a9e3f229a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:06.176820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:06.179678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:06.181710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 360/720] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:05 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:06.187951 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bb74c-f89c-42c7-b5ca-852a9e3f229a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:06.219753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d13957-df53-41a1-8048-63097c08ba17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:06.220440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d13957-df53-41a1-8048-63097c08ba17 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:06.220726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d13957-df53-41a1-8048-63097c08ba17 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:06.221005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d13957-df53-41a1-8048-63097c08ba17 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:06.221770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d13957-df53-41a1-8048-63097c08ba17 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:06.222080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d13957-df53-41a1-8048-63097c08ba17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:06.225048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d13957-df53-41a1-8048-63097c08ba17 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:06.234065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 361/721] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:13:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:06.243537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d13957-df53-41a1-8048-63097c08ba17 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:06.322067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bb74c-f89c-42c7-b5ca-852a9e3f229a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:06.336688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:06.350926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:06.361827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb27136-057f-4891-ba91-030fdc083763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:06.362858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27136-057f-4891-ba91-030fdc083763 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:06.363350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27136-057f-4891-ba91-030fdc083763 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:06.363867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb27136-057f-4891-ba91-030fdc083763 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:06.364775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb27136-057f-4891-ba91-030fdc083763 None admin] RBAC: Adding query filter params (name=network) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:06.365281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb27136-057f-4891-ba91-030fdc083763 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:06.366914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb27136-057f-4891-ba91-030fdc083763 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:06.372706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 361/722] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:06.387576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb27136-057f-4891-ba91-030fdc083763 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:06.423594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d13957-df53-41a1-8048-63097c08ba17 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:06.518147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb27136-057f-4891-ba91-030fdc083763 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:06.539881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfce524-391f-4c03-b94f-ab9415d2fe2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:06.541071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfce524-391f-4c03-b94f-ab9415d2fe2d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:06.541576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfce524-391f-4c03-b94f-ab9415d2fe2d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:06.542590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfce524-391f-4c03-b94f-ab9415d2fe2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:06.556074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98227e-93ec-4d09-b9ee-96b6c11f6ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:06.556746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98227e-93ec-4d09-b9ee-96b6c11f6ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:06.557026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98227e-93ec-4d09-b9ee-96b6c11f6ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:06.557388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98227e-93ec-4d09-b9ee-96b6c11f6ce3 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:06.558701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98227e-93ec-4d09-b9ee-96b6c11f6ce3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:06.559007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98227e-93ec-4d09-b9ee-96b6c11f6ce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:06.566571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98227e-93ec-4d09-b9ee-96b6c11f6ce3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:06.567447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 362/723] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:06.575773 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfce524-391f-4c03-b94f-ab9415d2fe2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:06.578025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfce524-391f-4c03-b94f-ab9415d2fe2d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ebfce524-391f-4c03-b94f-ab9415d2fe2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:06.613860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:06.614864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:06.615834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:06.615834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:06.615834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:06.615834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:06.616558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 362/724] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:06.622602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98227e-93ec-4d09-b9ee-96b6c11f6ce3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:06.672997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:06.761147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98227e-93ec-4d09-b9ee-96b6c11f6ce3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:06.810280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d11d3-cc1e-467f-ba88-e9d8005e696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:06.810997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d11d3-cc1e-467f-ba88-e9d8005e696a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:06.811282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d11d3-cc1e-467f-ba88-e9d8005e696a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:06.811551 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d11d3-cc1e-467f-ba88-e9d8005e696a None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:06.812288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d11d3-cc1e-467f-ba88-e9d8005e696a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:06.812591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d11d3-cc1e-467f-ba88-e9d8005e696a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:06.813988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d11d3-cc1e-467f-ba88-e9d8005e696a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:06.829542 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 363/725] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:06.834928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfce524-391f-4c03-b94f-ab9415d2fe2d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:06.947333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:06.952193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfce524-391f-4c03-b94f-ab9415d2fe2d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:06.994214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033c622-7f70-4288-98f5-5d03f07572f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:06.995087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033c622-7f70-4288-98f5-5d03f07572f1 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:06.995434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033c622-7f70-4288-98f5-5d03f07572f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:06.995720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033c622-7f70-4288-98f5-5d03f07572f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:07.004181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033c622-7f70-4288-98f5-5d03f07572f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:07.005643 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033c622-7f70-4288-98f5-5d03f07572f1 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-6033c622-7f70-4288-98f5-5d03f07572f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:07.030374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:07.031393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:07.032334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:07.032334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:07.032334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:07.032334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:07.032334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:07.032334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:07.032334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:07.032334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:07.032334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:07.032334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:07.032334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 363/726] 10.222.0.93 () {56 vars in 1117 bytes} [Fri Jun 3 15:13:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:07.039207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d11d3-cc1e-467f-ba88-e9d8005e696a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:07.175908 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:07.181292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d11d3-cc1e-467f-ba88-e9d8005e696a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:07.232421 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71720a-3e2f-4e7b-a8fb-5fb526213800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:07.233600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71720a-3e2f-4e7b-a8fb-5fb526213800 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:07.234097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71720a-3e2f-4e7b-a8fb-5fb526213800 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:07.234615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71720a-3e2f-4e7b-a8fb-5fb526213800 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:07.236274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71720a-3e2f-4e7b-a8fb-5fb526213800 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:07.236840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71720a-3e2f-4e7b-a8fb-5fb526213800 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:07.239585 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71720a-3e2f-4e7b-a8fb-5fb526213800 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:07.266491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 364/727] 10.222.0.93 () {56 vars in 1149 bytes} [Fri Jun 3 15:13:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:07.271939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033c622-7f70-4288-98f5-5d03f07572f1 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:07.380769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033c622-7f70-4288-98f5-5d03f07572f1 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:07.422037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865469db-dd0a-4a41-a3ae-5fa53c46362f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:07.422721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865469db-dd0a-4a41-a3ae-5fa53c46362f None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:07.423003 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865469db-dd0a-4a41-a3ae-5fa53c46362f None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:07.423272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865469db-dd0a-4a41-a3ae-5fa53c46362f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:07.457548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:07.519714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865469db-dd0a-4a41-a3ae-5fa53c46362f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:07.521191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865469db-dd0a-4a41-a3ae-5fa53c46362f None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:07.583743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1887871-dcec-4754-8022-7c8ef84e7f77 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:07.584387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1887871-dcec-4754-8022-7c8ef84e7f77 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:07.584387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1887871-dcec-4754-8022-7c8ef84e7f77 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:07.586118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 365/728] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:07.592127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e57403-08a3-478f-8853-e06ac11e17b0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:07.592411 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e57403-08a3-478f-8853-e06ac11e17b0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:07.592902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e57403-08a3-478f-8853-e06ac11e17b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:07.618700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-64e57403-08a3-478f-8853-e06ac11e17b0 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:07.619889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-64e57403-08a3-478f-8853-e06ac11e17b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:07.703503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 366/729] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:13:07.721881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 364/730] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 03 15:13:07.727580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865469db-dd0a-4a41-a3ae-5fa53c46362f None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:07.727947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e57403-08a3-478f-8853-e06ac11e17b0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:07.831247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:07.847184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e57403-08a3-478f-8853-e06ac11e17b0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:07.895069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:07.914521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ed191-e84e-4740-8fb7-5312c3ea1d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:07.915670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ed191-e84e-4740-8fb7-5312c3ea1d4d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:07.916019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ed191-e84e-4740-8fb7-5312c3ea1d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:07.916298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ed191-e84e-4740-8fb7-5312c3ea1d4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:07.928851 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ed191-e84e-4740-8fb7-5312c3ea1d4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:07.931461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ed191-e84e-4740-8fb7-5312c3ea1d4d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9b1ed191-e84e-4740-8fb7-5312c3ea1d4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:07.955126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:07.957255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:07.958422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:07.958422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:07.958422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:07.958422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:07.958422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 367/731] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:07.969103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ed191-e84e-4740-8fb7-5312c3ea1d4d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:07.975155 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865469db-dd0a-4a41-a3ae-5fa53c46362f None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:08.023844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6d613-3324-45b4-b8ce-fbb40e15ec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:08.024798 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6d613-3324-45b4-b8ce-fbb40e15ec7b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:08.025082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6d613-3324-45b4-b8ce-fbb40e15ec7b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:08.025359 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6d613-3324-45b4-b8ce-fbb40e15ec7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:08.046087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6d613-3324-45b4-b8ce-fbb40e15ec7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-f1c6d613-3324-45b4-b8ce-fbb40e15ec7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:08.067506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:08.068606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:08.069505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:08.073591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 365/732] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:08.081203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6d613-3324-45b4-b8ce-fbb40e15ec7b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:08.132016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ed191-e84e-4740-8fb7-5312c3ea1d4d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:08.184129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c713e8-0391-45e6-90b7-1342fb4999b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:08.184816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c713e8-0391-45e6-90b7-1342fb4999b5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:08.185103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c713e8-0391-45e6-90b7-1342fb4999b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:08.185381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c713e8-0391-45e6-90b7-1342fb4999b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:08.186505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c713e8-0391-45e6-90b7-1342fb4999b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:08.186901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c713e8-0391-45e6-90b7-1342fb4999b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:08.188450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c713e8-0391-45e6-90b7-1342fb4999b5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:08.199141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:08.201551 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 368/733] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:08.211647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c713e8-0391-45e6-90b7-1342fb4999b5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:08.391240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c713e8-0391-45e6-90b7-1342fb4999b5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:08.393240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6d613-3324-45b4-b8ce-fbb40e15ec7b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:08.421358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:08.450510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ed08f-6f3a-43c2-9276-a65c97bee81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:08.452574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed08f-6f3a-43c2-9276-a65c97bee81a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:08.453052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed08f-6f3a-43c2-9276-a65c97bee81a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:08.453510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed08f-6f3a-43c2-9276-a65c97bee81a None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:08.454413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ed08f-6f3a-43c2-9276-a65c97bee81a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:08.454763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ed08f-6f3a-43c2-9276-a65c97bee81a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:08.456191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ed08f-6f3a-43c2-9276-a65c97bee81a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:08.469940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb09a5-5750-4e4f-8306-e39080b2fe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:08.470781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb09a5-5750-4e4f-8306-e39080b2fe3c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:08.471067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb09a5-5750-4e4f-8306-e39080b2fe3c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:08.471345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb09a5-5750-4e4f-8306-e39080b2fe3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:08.478136 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 366/734] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:13:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:08.483835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ed08f-6f3a-43c2-9276-a65c97bee81a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:08.487362 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb09a5-5750-4e4f-8306-e39080b2fe3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:08.488845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb09a5-5750-4e4f-8306-e39080b2fe3c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c4fb09a5-5750-4e4f-8306-e39080b2fe3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:08.515588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:08.516689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:08.517804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:08.517804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:08.517804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:08.517804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:08.517804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:08.517804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:08.517804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:08.517804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:08.518640 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 369/735] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:08.527690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb09a5-5750-4e4f-8306-e39080b2fe3c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:08.622968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:08.685083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb09a5-5750-4e4f-8306-e39080b2fe3c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:08.689292 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ed08f-6f3a-43c2-9276-a65c97bee81a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:08.728354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e64f9-bc40-482d-9b26-c0dc61b1e1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:08.729024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e64f9-bc40-482d-9b26-c0dc61b1e1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:08.729320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e64f9-bc40-482d-9b26-c0dc61b1e1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:08.729597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e64f9-bc40-482d-9b26-c0dc61b1e1f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:08.730383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e64f9-bc40-482d-9b26-c0dc61b1e1f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:08.730721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e64f9-bc40-482d-9b26-c0dc61b1e1f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:08.732157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e64f9-bc40-482d-9b26-c0dc61b1e1f3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:08.735157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00164e-06ee-4db9-957c-699b60b66846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:08.736030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00164e-06ee-4db9-957c-699b60b66846 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:08.736315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00164e-06ee-4db9-957c-699b60b66846 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:08.736590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00164e-06ee-4db9-957c-699b60b66846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:08.754572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 370/736] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:08.754572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60365b4b-da26-4203-a936-0ab736d1f5df None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:08.754572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60365b4b-da26-4203-a936-0ab736d1f5df None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:08.754572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60365b4b-da26-4203-a936-0ab736d1f5df None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:08.754974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00164e-06ee-4db9-957c-699b60b66846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:08.755062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 371/737] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:08.755062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b972a7-0238-4ba4-b443-a191d9ae5d6e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:08.755062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b972a7-0238-4ba4-b443-a191d9ae5d6e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:08.755241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00164e-06ee-4db9-957c-699b60b66846 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:08.755310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b972a7-0238-4ba4-b443-a191d9ae5d6e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:08.755310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 372/738] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:08.755310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b972a7-0238-4ba4-b443-a191d9ae5d6e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-4a00164e-06ee-4db9-957c-699b60b66846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:08.779883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:08.780910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:08.782243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:08.782243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:08.782243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:08.782243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:08.782243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 367/739] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:08.783478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7de375-d0af-4af4-9a84-4d2c0c34bc4a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:08.783765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7de375-d0af-4af4-9a84-4d2c0c34bc4a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:08.784157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7de375-d0af-4af4-9a84-4d2c0c34bc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:08.802290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0f7de375-d0af-4af4-9a84-4d2c0c34bc4a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:08.803523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0f7de375-d0af-4af4-9a84-4d2c0c34bc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:08.832265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:08.872869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b972a7-0238-4ba4-b443-a191d9ae5d6e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:08.886678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 368/740] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:13:08.886678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b853-9b19-40ba-b174-4f60e6802455 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:08.886678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b853-9b19-40ba-b174-4f60e6802455 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:08.886678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7b853-9b19-40ba-b174-4f60e6802455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:08.900004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-3ce7b853-9b19-40ba-b174-4f60e6802455 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:08.903730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3ce7b853-9b19-40ba-b174-4f60e6802455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:08.908755 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:08.925337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94cfde8-e0b3-46ce-87ee-9c0e6dd182db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:08.925890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cfde8-e0b3-46ce-87ee-9c0e6dd182db None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:08.926507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cfde8-e0b3-46ce-87ee-9c0e6dd182db None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:08.926507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cfde8-e0b3-46ce-87ee-9c0e6dd182db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:08.927288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cfde8-e0b3-46ce-87ee-9c0e6dd182db None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=0603019b08d94980863f6612907601a9) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:08.927600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cfde8-e0b3-46ce-87ee-9c0e6dd182db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:08.929111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cfde8-e0b3-46ce-87ee-9c0e6dd182db None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:08.935721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 373/741] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:08 2022] GET /identity/v3/role_assignments?user.id=0603019b08d94980863f6612907601a9&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:08.936625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94cfde8-e0b3-46ce-87ee-9c0e6dd182db None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:09.002731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 369/742] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:13:09.003954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7b853-9b19-40ba-b174-4f60e6802455 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:09.098780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94cfde8-e0b3-46ce-87ee-9c0e6dd182db None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:09.128153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:09.135456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7b853-9b19-40ba-b174-4f60e6802455 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:09.140663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4f801-4138-4ff5-94ee-16b60a785908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:09.141559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4f801-4138-4ff5-94ee-16b60a785908 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:09.141990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4f801-4138-4ff5-94ee-16b60a785908 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:09.142446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4f801-4138-4ff5-94ee-16b60a785908 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:09.143359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4f801-4138-4ff5-94ee-16b60a785908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:09.143818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4f801-4138-4ff5-94ee-16b60a785908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:09.145531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4f801-4138-4ff5-94ee-16b60a785908 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:09.161605 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 374/743] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:09.163181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4f801-4138-4ff5-94ee-16b60a785908 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:09.191955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca61c8-2cd5-4f33-b2fc-9741a8a1a609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:09.193069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca61c8-2cd5-4f33-b2fc-9741a8a1a609 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:09.193362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca61c8-2cd5-4f33-b2fc-9741a8a1a609 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:09.193900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca61c8-2cd5-4f33-b2fc-9741a8a1a609 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:09.195129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca61c8-2cd5-4f33-b2fc-9741a8a1a609 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:09.196789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca61c8-2cd5-4f33-b2fc-9741a8a1a609 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:09.218100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 370/744] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:13:09 2022] POST /identity/v3/services => generated 251 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:09.219422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca61c8-2cd5-4f33-b2fc-9741a8a1a609 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:09.221908 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:09.297245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4f801-4138-4ff5-94ee-16b60a785908 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:09.381162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca61c8-2cd5-4f33-b2fc-9741a8a1a609 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:09.394742 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268076c-a4b3-4ca3-b4e4-caf9503c1fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:09.395628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268076c-a4b3-4ca3-b4e4-caf9503c1fcb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:09.395972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268076c-a4b3-4ca3-b4e4-caf9503c1fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:09.396255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268076c-a4b3-4ca3-b4e4-caf9503c1fcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:09.419810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268076c-a4b3-4ca3-b4e4-caf9503c1fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d268076c-a4b3-4ca3-b4e4-caf9503c1fcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:09.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:09.435729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:09.436582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:09.442345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 375/745] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:09.444100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e720-aa5f-4744-9e21-17dcdeb6e841 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:09.444634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e720-aa5f-4744-9e21-17dcdeb6e841 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:09.445147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e720-aa5f-4744-9e21-17dcdeb6e841 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:09.447249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 376/746] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:09.448783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6e720-aa5f-4744-9e21-17dcdeb6e841 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:09.455822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e44b45-4ed9-4ebc-8dcb-42345ca44d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:09.456652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e44b45-4ed9-4ebc-8dcb-42345ca44d89 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:09.456934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e44b45-4ed9-4ebc-8dcb-42345ca44d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:09.457233 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e44b45-4ed9-4ebc-8dcb-42345ca44d89 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:09.467002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e44b45-4ed9-4ebc-8dcb-42345ca44d89 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:09.468711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e44b45-4ed9-4ebc-8dcb-42345ca44d89 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-78e44b45-4ed9-4ebc-8dcb-42345ca44d89 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:09.486053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:09.487129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:09.488084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:09.488084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:09.488084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:09.488084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:09.488084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:09.488084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:09.488084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:09.488084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:09.488084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:13:09.488084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:09.488084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 371/747] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:09.489172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddcf2d-46a1-4e73-b07c-854b467423b4 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:09.489273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddcf2d-46a1-4e73-b07c-854b467423b4 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:09.489516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddcf2d-46a1-4e73-b07c-854b467423b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:09.514627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-f4ddcf2d-46a1-4e73-b07c-854b467423b4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:09.515006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f4ddcf2d-46a1-4e73-b07c-854b467423b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:09.518407 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:09.592604 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6e720-aa5f-4744-9e21-17dcdeb6e841 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:09.597591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:09.606115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 372/748] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 03 15:13:09.606864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddcf2d-46a1-4e73-b07c-854b467423b4 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:09.665992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f6af0-9807-4572-823b-896ef64fa0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:09.668015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f6af0-9807-4572-823b-896ef64fa0bf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:09.668462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f6af0-9807-4572-823b-896ef64fa0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:09.668926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f6af0-9807-4572-823b-896ef64fa0bf None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:09.670090 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f6af0-9807-4572-823b-896ef64fa0bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:09.670566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f6af0-9807-4572-823b-896ef64fa0bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:09.672824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f6af0-9807-4572-823b-896ef64fa0bf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:09.686610 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 377/749] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:13:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:09.688327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f6af0-9807-4572-823b-896ef64fa0bf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:09.732000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddcf2d-46a1-4e73-b07c-854b467423b4 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:09.770072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf340f-b187-4527-be18-f9d98ac5bca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:09.771025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf340f-b187-4527-be18-f9d98ac5bca9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:09.771483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf340f-b187-4527-be18-f9d98ac5bca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:09.771900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf340f-b187-4527-be18-f9d98ac5bca9 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:09.772878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf340f-b187-4527-be18-f9d98ac5bca9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:09.773344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf340f-b187-4527-be18-f9d98ac5bca9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:09.775089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf340f-b187-4527-be18-f9d98ac5bca9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:09.786007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:09.798345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 373/750] 10.222.0.93 () {56 vars in 1135 bytes} [Fri Jun 3 15:13:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:09.799522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf340f-b187-4527-be18-f9d98ac5bca9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:09.817078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f6af0-9807-4572-823b-896ef64fa0bf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:09.857571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a54c08-ee07-4b65-a781-fdf1dcb5635d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:09.858399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a54c08-ee07-4b65-a781-fdf1dcb5635d None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:09.858808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a54c08-ee07-4b65-a781-fdf1dcb5635d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:09.859183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a54c08-ee07-4b65-a781-fdf1dcb5635d None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:09.860027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a54c08-ee07-4b65-a781-fdf1dcb5635d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:09.861618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a54c08-ee07-4b65-a781-fdf1dcb5635d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:09.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 378/751] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:13:09 2022] POST /identity/v3/services => generated 240 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:09.894558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a54c08-ee07-4b65-a781-fdf1dcb5635d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:09.946151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf340f-b187-4527-be18-f9d98ac5bca9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:09.951145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:10.023007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38fbb-becb-48a2-914d-b52e76321539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:10.026713 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38fbb-becb-48a2-914d-b52e76321539 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:10.028016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38fbb-becb-48a2-914d-b52e76321539 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:10.028680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed38fbb-becb-48a2-914d-b52e76321539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:10.050259 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed38fbb-becb-48a2-914d-b52e76321539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:10.058443 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed38fbb-becb-48a2-914d-b52e76321539 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-3ed38fbb-becb-48a2-914d-b52e76321539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:10.087034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:10.088518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:10.090076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:10.090076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:10.090076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:10.090076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:10.094604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 374/752] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:10.100855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38fbb-becb-48a2-914d-b52e76321539 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:10.125088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a54c08-ee07-4b65-a781-fdf1dcb5635d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:10.174314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ec590-4f35-4ed6-bfad-a472365bd03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:10.175051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ec590-4f35-4ed6-bfad-a472365bd03f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:10.175391 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ec590-4f35-4ed6-bfad-a472365bd03f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:10.175603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ec590-4f35-4ed6-bfad-a472365bd03f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:10.178936 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:10.271134 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed38fbb-becb-48a2-914d-b52e76321539 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:10.274441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ec590-4f35-4ed6-bfad-a472365bd03f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:10.276353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ec590-4f35-4ed6-bfad-a472365bd03f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:10.313749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:10.314918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051a8c2-0a89-476b-851a-9379d584e2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:10.315958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051a8c2-0a89-476b-851a-9379d584e2ee None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:10.316326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051a8c2-0a89-476b-851a-9379d584e2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:10.316746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051a8c2-0a89-476b-851a-9379d584e2ee None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:10.317735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051a8c2-0a89-476b-851a-9379d584e2ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:10.318051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051a8c2-0a89-476b-851a-9379d584e2ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:10.319484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4051a8c2-0a89-476b-851a-9379d584e2ee None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:10.340639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 375/753] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:10.345815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051a8c2-0a89-476b-851a-9379d584e2ee None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:10.401733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 379/754] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:13:10.407885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ec590-4f35-4ed6-bfad-a472365bd03f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:10.527014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ec590-4f35-4ed6-bfad-a472365bd03f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:10.580921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051a8c2-0a89-476b-851a-9379d584e2ee None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:10.589577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:10.601402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:10.610504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45aadf-9fc1-4d7d-a75a-037425d2e554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:10.611656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45aadf-9fc1-4d7d-a75a-037425d2e554 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:10.612165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45aadf-9fc1-4d7d-a75a-037425d2e554 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:10.612891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45aadf-9fc1-4d7d-a75a-037425d2e554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:10.645784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b4fe9-c471-4818-b9fe-52537076c966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:10.646295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45aadf-9fc1-4d7d-a75a-037425d2e554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:10.647225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b4fe9-c471-4818-b9fe-52537076c966 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:10.647742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b4fe9-c471-4818-b9fe-52537076c966 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:10.647848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45aadf-9fc1-4d7d-a75a-037425d2e554 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:10.648336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b4fe9-c471-4818-b9fe-52537076c966 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:10.664856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b4fe9-c471-4818-b9fe-52537076c966 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:10.674561 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b4fe9-c471-4818-b9fe-52537076c966 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-f50b4fe9-c471-4818-b9fe-52537076c966 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:10.691860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:10.692927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:10.693881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:10.693881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:10.693881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:10.693881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:10.693881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:10.693881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:10.693881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:10.693881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:10.693881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:13:10.693881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-6f45aadf-9fc1-4d7d-a75a-037425d2e554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:10.695733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:10.696963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:10.704204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:10.704204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:10.704204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:10.704204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:10.704204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 380/755] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:10.704204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45aadf-9fc1-4d7d-a75a-037425d2e554 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:10.707052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 376/756] 10.222.0.93 () {56 vars in 1093 bytes} [Fri Jun 3 15:13:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:10.712336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b4fe9-c471-4818-b9fe-52537076c966 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:10.821207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45aadf-9fc1-4d7d-a75a-037425d2e554 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:10.865877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e15451-921e-4325-a70c-3bf41fcf2daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:10.866558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e15451-921e-4325-a70c-3bf41fcf2daf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:10.867077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e15451-921e-4325-a70c-3bf41fcf2daf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:10.867357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e15451-921e-4325-a70c-3bf41fcf2daf None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:10.868081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e15451-921e-4325-a70c-3bf41fcf2daf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:10.868382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e15451-921e-4325-a70c-3bf41fcf2daf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:10.869758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e15451-921e-4325-a70c-3bf41fcf2daf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:10.890574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 381/757] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:10.897083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e15451-921e-4325-a70c-3bf41fcf2daf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:10.959476 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b4fe9-c471-4818-b9fe-52537076c966 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:10.974931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:11.029392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77329d1f-94bd-4fcb-bdb6-9da452b932c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:11.030096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77329d1f-94bd-4fcb-bdb6-9da452b932c4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:11.030378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77329d1f-94bd-4fcb-bdb6-9da452b932c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:11.030678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77329d1f-94bd-4fcb-bdb6-9da452b932c4 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:11.031520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77329d1f-94bd-4fcb-bdb6-9da452b932c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:11.031829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77329d1f-94bd-4fcb-bdb6-9da452b932c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:11.033270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77329d1f-94bd-4fcb-bdb6-9da452b932c4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:11.072664 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 377/758] 10.222.0.93 () {56 vars in 1133 bytes} [Fri Jun 3 15:13:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:11.078704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77329d1f-94bd-4fcb-bdb6-9da452b932c4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:11.097755 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e15451-921e-4325-a70c-3bf41fcf2daf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:11.121656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:11.180039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35742a69-3663-42c8-992d-d92956d49c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:11.180842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742a69-3663-42c8-992d-d92956d49c0a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:11.181121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742a69-3663-42c8-992d-d92956d49c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:11.181687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742a69-3663-42c8-992d-d92956d49c0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:11.203224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35742a69-3663-42c8-992d-d92956d49c0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:11.204722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35742a69-3663-42c8-992d-d92956d49c0a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-35742a69-3663-42c8-992d-d92956d49c0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:11.230323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:11.231341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:11.232245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:11.232245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:11.232245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:11.232245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:11.232245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:11.232245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:11.232245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:11.232245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:11.233103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 382/759] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:11.241879 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35742a69-3663-42c8-992d-d92956d49c0a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:11.264744 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77329d1f-94bd-4fcb-bdb6-9da452b932c4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:11.354113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:11.359991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35742a69-3663-42c8-992d-d92956d49c0a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:11.365315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a46b7-fde4-4688-a29b-32d3ff1e0624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:11.366139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a46b7-fde4-4688-a29b-32d3ff1e0624 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:11.366423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a46b7-fde4-4688-a29b-32d3ff1e0624 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:11.366738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a46b7-fde4-4688-a29b-32d3ff1e0624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:11.410244 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a46b7-fde4-4688-a29b-32d3ff1e0624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:11.410244 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a46b7-fde4-4688-a29b-32d3ff1e0624 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:11.422764 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b61832-b736-4633-bb6a-88b48745d100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:11.424140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b61832-b736-4633-bb6a-88b48745d100 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:11.424848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b61832-b736-4633-bb6a-88b48745d100 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:11.425496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b61832-b736-4633-bb6a-88b48745d100 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:11.431616 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b61832-b736-4633-bb6a-88b48745d100 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:11.432796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b61832-b736-4633-bb6a-88b48745d100 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:11.443173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b61832-b736-4633-bb6a-88b48745d100 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:11.475081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 383/760] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ca8a46b7-fde4-4688-a29b-32d3ff1e0624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:11.482553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:11.483496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:11.484415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:11.484415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:11.484415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:11.484415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:11.484415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 378/761] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:11.484695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b61832-b736-4633-bb6a-88b48745d100 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:11.488270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a46b7-fde4-4688-a29b-32d3ff1e0624 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:11.736004 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b61832-b736-4633-bb6a-88b48745d100 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:11.742315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:11.802197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a46b7-fde4-4688-a29b-32d3ff1e0624 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:11.823319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4b8aa-9bdc-43dd-b14e-d55b63202c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:11.824455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4b8aa-9bdc-43dd-b14e-d55b63202c0c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:11.824945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4b8aa-9bdc-43dd-b14e-d55b63202c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:11.825423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4b8aa-9bdc-43dd-b14e-d55b63202c0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:11.826669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4b8aa-9bdc-43dd-b14e-d55b63202c0c None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=23546323ca33412fb388afe308a97499, user.id=bb4fb8f859dc4489b8c4fd8fdc09c3f6) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:11.827192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4b8aa-9bdc-43dd-b14e-d55b63202c0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:11.828887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4b8aa-9bdc-43dd-b14e-d55b63202c0c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:11.841042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 384/762] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:11 2022] GET /identity/v3/role_assignments?user.id=bb4fb8f859dc4489b8c4fd8fdc09c3f6&scope.project.id=23546323ca33412fb388afe308a97499&role.id=d67707782358417ba9f25b2793ca2890 => generated 259 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:11.843502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7441dc-bf5c-451d-9fc5-8c01ba09d645 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:11.844243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7441dc-bf5c-451d-9fc5-8c01ba09d645 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:11.844896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7441dc-bf5c-451d-9fc5-8c01ba09d645 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:11.853434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 385/763] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:11.857039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247fbfb-d792-403e-911f-7662e0569a6b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:11.857894 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247fbfb-d792-403e-911f-7662e0569a6b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:11.858798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247fbfb-d792-403e-911f-7662e0569a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:11.915621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819fc62-b220-4092-8c67-ca3dc78341c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:11.916360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819fc62-b220-4092-8c67-ca3dc78341c9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:11.917156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819fc62-b220-4092-8c67-ca3dc78341c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:11.917443 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819fc62-b220-4092-8c67-ca3dc78341c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:11.918760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819fc62-b220-4092-8c67-ca3dc78341c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:11.919088 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819fc62-b220-4092-8c67-ca3dc78341c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:11.920531 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819fc62-b220-4092-8c67-ca3dc78341c9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:11.929762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b247fbfb-d792-403e-911f-7662e0569a6b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:11.937543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b247fbfb-d792-403e-911f-7662e0569a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:11.962768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 379/764] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 475 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:11.971024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819fc62-b220-4092-8c67-ca3dc78341c9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:11.986781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:12.081332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 386/765] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:12.090563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33490069-43ea-4378-9874-ea40c7e02b83 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:12.090563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33490069-43ea-4378-9874-ea40c7e02b83 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:12.090563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33490069-43ea-4378-9874-ea40c7e02b83 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:12.090563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 387/766] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:12.092651 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a90d5-55ba-4a22-bd55-1124ad044ae5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:12.092901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a90d5-55ba-4a22-bd55-1124ad044ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:12.093184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a90d5-55ba-4a22-bd55-1124ad044ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:12.120482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b55a90d5-55ba-4a22-bd55-1124ad044ae5 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:12.121730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b55a90d5-55ba-4a22-bd55-1124ad044ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:12.196094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819fc62-b220-4092-8c67-ca3dc78341c9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:12.230652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 388/767] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:12.232365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a90d5-55ba-4a22-bd55-1124ad044ae5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:12.256537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73baf9-8cb4-4991-9f0b-56fa9719218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:12.257365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73baf9-8cb4-4991-9f0b-56fa9719218e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:12.257649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73baf9-8cb4-4991-9f0b-56fa9719218e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:12.257926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73baf9-8cb4-4991-9f0b-56fa9719218e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:12.266143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73baf9-8cb4-4991-9f0b-56fa9719218e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:12.267718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73baf9-8cb4-4991-9f0b-56fa9719218e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:12.268586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-7f73baf9-8cb4-4991-9f0b-56fa9719218e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:12.289082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:12.290066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:12.291357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:12.291357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:12.291357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:12.291357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:12.291357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:12.291357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:12.291357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:12.291357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:12.292506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 380/768] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:12.293097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73baf9-8cb4-4991-9f0b-56fa9719218e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:12.351975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:12.379964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a90d5-55ba-4a22-bd55-1124ad044ae5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:12.446484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73baf9-8cb4-4991-9f0b-56fa9719218e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:12.451850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42c17c-99d8-4d15-8961-0538ebee0c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:12.452653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c17c-99d8-4d15-8961-0538ebee0c93 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:12.452930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c17c-99d8-4d15-8961-0538ebee0c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:12.453199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42c17c-99d8-4d15-8961-0538ebee0c93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:12.477953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c42c17c-99d8-4d15-8961-0538ebee0c93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2c42c17c-99d8-4d15-8961-0538ebee0c93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:12.486717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:12.487699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:12.488583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 389/769] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:13:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:12.490194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42c17c-99d8-4d15-8961-0538ebee0c93 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:12.498363 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cd147-6461-4fac-9138-c860bdf9f934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:12.501265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cd147-6461-4fac-9138-c860bdf9f934 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:12.503611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cd147-6461-4fac-9138-c860bdf9f934 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:12.504538 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309cd147-6461-4fac-9138-c860bdf9f934 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:12.507317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cd147-6461-4fac-9138-c860bdf9f934 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:12.517083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309cd147-6461-4fac-9138-c860bdf9f934 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-309cd147-6461-4fac-9138-c860bdf9f934 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:12.533550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:12.535283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 15:13:12.536919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:12.546034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 381/770] 10.222.0.93 () {56 vars in 1129 bytes} [Fri Jun 3 15:13:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:12.547351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cd147-6461-4fac-9138-c860bdf9f934 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:12.616313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:12.616782 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42c17c-99d8-4d15-8961-0538ebee0c93 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:12.667423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869777e-7060-4bbe-a3e8-f1769b8a651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:12.669213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869777e-7060-4bbe-a3e8-f1769b8a651d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:12.669213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869777e-7060-4bbe-a3e8-f1769b8a651d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:12.674552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869777e-7060-4bbe-a3e8-f1769b8a651d None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:12.674552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869777e-7060-4bbe-a3e8-f1769b8a651d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:12.674552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869777e-7060-4bbe-a3e8-f1769b8a651d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:12.678011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309cd147-6461-4fac-9138-c860bdf9f934 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:12.679259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869777e-7060-4bbe-a3e8-f1769b8a651d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:12.696249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 390/771] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:12.697466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869777e-7060-4bbe-a3e8-f1769b8a651d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:12.701436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:12.741868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551b425-8c72-4f95-9a90-3f8b447eb42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:12.743537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551b425-8c72-4f95-9a90-3f8b447eb42e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:12.744783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551b425-8c72-4f95-9a90-3f8b447eb42e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:12.746100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551b425-8c72-4f95-9a90-3f8b447eb42e None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:12.747130 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551b425-8c72-4f95-9a90-3f8b447eb42e None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:12.747927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551b425-8c72-4f95-9a90-3f8b447eb42e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:12.750216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551b425-8c72-4f95-9a90-3f8b447eb42e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:12.761072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 382/772] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:13:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:12.762292 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551b425-8c72-4f95-9a90-3f8b447eb42e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:12.833972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869777e-7060-4bbe-a3e8-f1769b8a651d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:12.888351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935404e0-a79a-41c8-88fc-511b84fa01a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:12.889336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935404e0-a79a-41c8-88fc-511b84fa01a7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:12.889697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935404e0-a79a-41c8-88fc-511b84fa01a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:12.890049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935404e0-a79a-41c8-88fc-511b84fa01a7 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:12.890883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935404e0-a79a-41c8-88fc-511b84fa01a7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:12.891597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935404e0-a79a-41c8-88fc-511b84fa01a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:12.892803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935404e0-a79a-41c8-88fc-511b84fa01a7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:12.897935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 391/773] 10.222.0.93 () {56 vars in 1124 bytes} [Fri Jun 3 15:13:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:12.903817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935404e0-a79a-41c8-88fc-511b84fa01a7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:12.912371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551b425-8c72-4f95-9a90-3f8b447eb42e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:12.946447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:12.966391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7aba33-2e2e-4bf9-b872-4aaa91c813bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:12.968685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aba33-2e2e-4bf9-b872-4aaa91c813bb None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:12.968685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aba33-2e2e-4bf9-b872-4aaa91c813bb None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:12.968685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7aba33-2e2e-4bf9-b872-4aaa91c813bb None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/c1966524091242e7a6cd2758027a0d4f/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:12.969962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7aba33-2e2e-4bf9-b872-4aaa91c813bb None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=c1966524091242e7a6cd2758027a0d4f, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:12.971541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7aba33-2e2e-4bf9-b872-4aaa91c813bb None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:12.991088 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 383/774] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:13:12 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/c1966524091242e7a6cd2758027a0d4f/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:13:12.991474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:12.992683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef46d6-789a-4c1f-81b7-fcb7ca795331 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:12.993088 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef46d6-789a-4c1f-81b7-fcb7ca795331 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:12.993262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef46d6-789a-4c1f-81b7-fcb7ca795331 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:12.995737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 384/775] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:13.002732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef46d6-789a-4c1f-81b7-fcb7ca795331 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:13.078429 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935404e0-a79a-41c8-88fc-511b84fa01a7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:13.129835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f323e07a-c89f-48c9-a491-13b03d1a58ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:13.130887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323e07a-c89f-48c9-a491-13b03d1a58ca None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:13.131305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323e07a-c89f-48c9-a491-13b03d1a58ca None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:13.131589 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f323e07a-c89f-48c9-a491-13b03d1a58ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:13.146601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323e07a-c89f-48c9-a491-13b03d1a58ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:13.148654 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f323e07a-c89f-48c9-a491-13b03d1a58ca None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f323e07a-c89f-48c9-a491-13b03d1a58ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:13.174357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:13.175354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:13.176246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:13.176246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:13.176246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:13.176246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:13.177577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 392/776] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:13.178319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ddda3-bdba-4d13-9858-8038afc90514 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:13.178821 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ddda3-bdba-4d13-9858-8038afc90514 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:13.179224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ddda3-bdba-4d13-9858-8038afc90514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:13.188413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef46d6-789a-4c1f-81b7-fcb7ca795331 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:13.200947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1a8ddda3-bdba-4d13-9858-8038afc90514 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:13.203044 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1a8ddda3-bdba-4d13-9858-8038afc90514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:13.253214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166773d-4d09-4063-81eb-68a7344ecc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:13.254028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166773d-4d09-4063-81eb-68a7344ecc9f None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:13.254393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166773d-4d09-4063-81eb-68a7344ecc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:13.254795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8166773d-4d09-4063-81eb-68a7344ecc9f None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:13.255985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166773d-4d09-4063-81eb-68a7344ecc9f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:13.255985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166773d-4d09-4063-81eb-68a7344ecc9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:13.258371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8166773d-4d09-4063-81eb-68a7344ecc9f None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:13.272065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 385/777] 10.222.0.93 () {56 vars in 1124 bytes} [Fri Jun 3 15:13:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:13.273510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166773d-4d09-4063-81eb-68a7344ecc9f None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:13.315076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 393/778] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:13.316623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ddda3-bdba-4d13-9858-8038afc90514 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:13.329599 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:13.393904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8166773d-4d09-4063-81eb-68a7344ecc9f None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:13.394820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:13.447223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543da8a-0cd0-4de2-b44e-156701a52018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:13.448624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543da8a-0cd0-4de2-b44e-156701a52018 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:13.449073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543da8a-0cd0-4de2-b44e-156701a52018 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:13.449359 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543da8a-0cd0-4de2-b44e-156701a52018 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:13.450629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543da8a-0cd0-4de2-b44e-156701a52018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:13.450953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543da8a-0cd0-4de2-b44e-156701a52018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:13.452442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543da8a-0cd0-4de2-b44e-156701a52018 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:13.466774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 386/779] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:13.467670 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543da8a-0cd0-4de2-b44e-156701a52018 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:13.475680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ddda3-bdba-4d13-9858-8038afc90514 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:13.523119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7443e9d-2a21-40f0-9a3f-e05c2e9e1eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:13.523812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7443e9d-2a21-40f0-9a3f-e05c2e9e1eef None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:13.524240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7443e9d-2a21-40f0-9a3f-e05c2e9e1eef None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:13.524520 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7443e9d-2a21-40f0-9a3f-e05c2e9e1eef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:13.525411 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7443e9d-2a21-40f0-9a3f-e05c2e9e1eef None admin] RBAC: Adding query filter params (interface=public, service_id=50a4b9656c0b4636a99523094e60bb82, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:13.525729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7443e9d-2a21-40f0-9a3f-e05c2e9e1eef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:13.527739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7443e9d-2a21-40f0-9a3f-e05c2e9e1eef None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:13.539368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 394/780] 10.222.0.93 () {56 vars in 1251 bytes} [Fri Jun 3 15:13:13 2022] GET /identity/v3/endpoints?service_id=50a4b9656c0b4636a99523094e60bb82&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:13.540278 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7443e9d-2a21-40f0-9a3f-e05c2e9e1eef None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:13.602426 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543da8a-0cd0-4de2-b44e-156701a52018 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:13.610873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:13.642465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c76a23-3c55-4bff-8d13-8de861909b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:13.644191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c76a23-3c55-4bff-8d13-8de861909b6c None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:13.644642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c76a23-3c55-4bff-8d13-8de861909b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:13.644916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c76a23-3c55-4bff-8d13-8de861909b6c None admin] PATH_INFO: `/v3/services/network` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:13.645589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c76a23-3c55-4bff-8d13-8de861909b6c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:13.647030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c76a23-3c55-4bff-8d13-8de861909b6c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:13.657407 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7443e9d-2a21-40f0-9a3f-e05c2e9e1eef None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-68c76a23-3c55-4bff-8d13-8de861909b6c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:13.661960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:13.663279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:13.664307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 387/781] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:13 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:13.665144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c76a23-3c55-4bff-8d13-8de861909b6c None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:13.677772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:13.709884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71d9e7-e860-4e85-8c1b-fa2331245ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:13.710988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71d9e7-e860-4e85-8c1b-fa2331245ced None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:13.711455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71d9e7-e860-4e85-8c1b-fa2331245ced None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:13.712153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71d9e7-e860-4e85-8c1b-fa2331245ced None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:13.721422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71d9e7-e860-4e85-8c1b-fa2331245ced None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:13.723138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71d9e7-e860-4e85-8c1b-fa2331245ced None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7f71d9e7-e860-4e85-8c1b-fa2331245ced None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:13.741615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:13.743109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:13.744461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:13.744461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:13.744461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:13.744461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:13.744461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:13.744461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:13.744461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:13.744461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:13.744461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:13.744461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:13.747216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 395/782] 10.222.0.93 () {56 vars in 1117 bytes} [Fri Jun 3 15:13:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:13.748339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71d9e7-e860-4e85-8c1b-fa2331245ced None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:13.804255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c76a23-3c55-4bff-8d13-8de861909b6c None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:13.848147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf31b9-deeb-462e-891b-6dbef13c8d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:13.850035 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf31b9-deeb-462e-891b-6dbef13c8d96 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:13.850866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf31b9-deeb-462e-891b-6dbef13c8d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:13.851631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf31b9-deeb-462e-891b-6dbef13c8d96 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:13.853239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf31b9-deeb-462e-891b-6dbef13c8d96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:13.855550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daf31b9-deeb-462e-891b-6dbef13c8d96 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:13.870870 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 388/783] 10.222.0.93 () {56 vars in 1087 bytes} [Fri Jun 3 15:13:13 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:13.872213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf31b9-deeb-462e-891b-6dbef13c8d96 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:13.881975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71d9e7-e860-4e85-8c1b-fa2331245ced None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:13.961660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:13.964743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560b3cd-5e7f-4b4a-b2f6-74a24b423a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:13.965444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560b3cd-5e7f-4b4a-b2f6-74a24b423a8e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:13.965731 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560b3cd-5e7f-4b4a-b2f6-74a24b423a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:13.966010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560b3cd-5e7f-4b4a-b2f6-74a24b423a8e None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:13.966762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560b3cd-5e7f-4b4a-b2f6-74a24b423a8e None admin] RBAC: Adding query filter params (name=network) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:13.967072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560b3cd-5e7f-4b4a-b2f6-74a24b423a8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:13.968486 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560b3cd-5e7f-4b4a-b2f6-74a24b423a8e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:13.974234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 396/784] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:13.980457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560b3cd-5e7f-4b4a-b2f6-74a24b423a8e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:14.027763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:14.117628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf31b9-deeb-462e-891b-6dbef13c8d96 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:14.121042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560b3cd-5e7f-4b4a-b2f6-74a24b423a8e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:14.169072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ad71f-d6e7-476f-9744-1ecbc2494085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:14.169923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ad71f-d6e7-476f-9744-1ecbc2494085 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:14.171141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ad71f-d6e7-476f-9744-1ecbc2494085 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:14.171543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ad71f-d6e7-476f-9744-1ecbc2494085 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:14.172572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ad71f-d6e7-476f-9744-1ecbc2494085 None admin] RBAC: Adding query filter params (type=network) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:14.172991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ad71f-d6e7-476f-9744-1ecbc2494085 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:14.174593 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ad71f-d6e7-476f-9744-1ecbc2494085 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:14.182026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 397/785] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:14.190935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ad71f-d6e7-476f-9744-1ecbc2494085 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:14.202340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204484ae-4cc3-4463-81de-5006ab69b392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:14.203331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204484ae-4cc3-4463-81de-5006ab69b392 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:14.205328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204484ae-4cc3-4463-81de-5006ab69b392 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:14.205788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204484ae-4cc3-4463-81de-5006ab69b392 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:14.206982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204484ae-4cc3-4463-81de-5006ab69b392 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:14.207507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204484ae-4cc3-4463-81de-5006ab69b392 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:14.209236 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204484ae-4cc3-4463-81de-5006ab69b392 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:14.233048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 389/786] 10.222.0.93 () {56 vars in 1149 bytes} [Fri Jun 3 15:13:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:14.243574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204484ae-4cc3-4463-81de-5006ab69b392 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:14.356610 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:14.362304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ad71f-d6e7-476f-9744-1ecbc2494085 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:14.392489 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:14.409268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f369c18-7fef-4b3c-9620-0532115f6ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:14.409711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204484ae-4cc3-4463-81de-5006ab69b392 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:14.410507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f369c18-7fef-4b3c-9620-0532115f6ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:14.411040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f369c18-7fef-4b3c-9620-0532115f6ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:14.411724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f369c18-7fef-4b3c-9620-0532115f6ec5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:14.412958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f369c18-7fef-4b3c-9620-0532115f6ec5 None admin] RBAC: Adding query filter params (interface=public, service_id=d9f792e309ce43ccb7c761c89a0caacf, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:14.413512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f369c18-7fef-4b3c-9620-0532115f6ec5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:14.416309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f369c18-7fef-4b3c-9620-0532115f6ec5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:14.423019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 398/787] 10.222.0.93 () {56 vars in 1251 bytes} [Fri Jun 3 15:13:14 2022] GET /identity/v3/endpoints?service_id=d9f792e309ce43ccb7c761c89a0caacf&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:14.429597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f369c18-7fef-4b3c-9620-0532115f6ec5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:14.471868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0358b39-1a33-4d81-aefa-54db4a2714e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:14.473315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0358b39-1a33-4d81-aefa-54db4a2714e3 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:14.473688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0358b39-1a33-4d81-aefa-54db4a2714e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:14.474142 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0358b39-1a33-4d81-aefa-54db4a2714e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:14.489080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0358b39-1a33-4d81-aefa-54db4a2714e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:14.491316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0358b39-1a33-4d81-aefa-54db4a2714e3 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-e0358b39-1a33-4d81-aefa-54db4a2714e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:14.516771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:14.517739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:14.518730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:14.518730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:14.518730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:14.518730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:14.522641 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 390/788] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:14.524266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce35fcd-011d-474e-869d-36dcacfde31a None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:14.524550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce35fcd-011d-474e-869d-36dcacfde31a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:14.524825 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce35fcd-011d-474e-869d-36dcacfde31a None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:14.526405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 391/789] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:14.528563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce35fcd-011d-474e-869d-36dcacfde31a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:14.570499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f369c18-7fef-4b3c-9620-0532115f6ec5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:14.623880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565d51d-aeba-4185-be98-cddc0fec144a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:14.624996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d51d-aeba-4185-be98-cddc0fec144a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:14.625558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d51d-aeba-4185-be98-cddc0fec144a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:14.626118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565d51d-aeba-4185-be98-cddc0fec144a None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:14.627090 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565d51d-aeba-4185-be98-cddc0fec144a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:14.629069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565d51d-aeba-4185-be98-cddc0fec144a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:14.643261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 399/790] 10.222.0.93 () {56 vars in 1087 bytes} [Fri Jun 3 15:13:14 2022] GET /identity/v3/services => generated 772 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:14.645149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee396c-65a1-4134-aa69-c2aa62e6f0b5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:14.645763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee396c-65a1-4134-aa69-c2aa62e6f0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:14.646243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee396c-65a1-4134-aa69-c2aa62e6f0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:14.676398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce35fcd-011d-474e-869d-36dcacfde31a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:14.683503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8bee396c-65a1-4134-aa69-c2aa62e6f0b5 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:14.684743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8bee396c-65a1-4134-aa69-c2aa62e6f0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:14.739206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:14.762109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7d9f2-e0c3-4e35-a045-2e80527f1491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:14.763222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7d9f2-e0c3-4e35-a045-2e80527f1491 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:14.765393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7d9f2-e0c3-4e35-a045-2e80527f1491 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:14.765887 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7d9f2-e0c3-4e35-a045-2e80527f1491 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:14.770495 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7d9f2-e0c3-4e35-a045-2e80527f1491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:14.770833 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7d9f2-e0c3-4e35-a045-2e80527f1491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:14.773124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7d9f2-e0c3-4e35-a045-2e80527f1491 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:14.806340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 392/791] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:14.811657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7d9f2-e0c3-4e35-a045-2e80527f1491 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:14.817504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:14.836432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 400/792] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:14.871877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee396c-65a1-4134-aa69-c2aa62e6f0b5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:14.998216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7d9f2-e0c3-4e35-a045-2e80527f1491 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:15.038201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee396c-65a1-4134-aa69-c2aa62e6f0b5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:15.044062 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd4ca1-d2ad-4f93-b9f4-de54dddf68b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:15.044871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd4ca1-d2ad-4f93-b9f4-de54dddf68b0 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:15.045156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd4ca1-d2ad-4f93-b9f4-de54dddf68b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:15.045437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd4ca1-d2ad-4f93-b9f4-de54dddf68b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:15.053562 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd4ca1-d2ad-4f93-b9f4-de54dddf68b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:15.054428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd4ca1-d2ad-4f93-b9f4-de54dddf68b0 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-badd4ca1-d2ad-4f93-b9f4-de54dddf68b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:15.069874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:15.075125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:15.076547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:15.076547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:15.076547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:15.076547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:15.076547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:15.076547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:15.076547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:15.076547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:15.076547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 393/793] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:15.080335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd4ca1-d2ad-4f93-b9f4-de54dddf68b0 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:15.101794 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cef204-4499-47c3-8d16-6241a0c1a1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:15.102908 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cef204-4499-47c3-8d16-6241a0c1a1ec None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:15.103369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cef204-4499-47c3-8d16-6241a0c1a1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:15.103798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cef204-4499-47c3-8d16-6241a0c1a1ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:15.127984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cef204-4499-47c3-8d16-6241a0c1a1ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a8cef204-4499-47c3-8d16-6241a0c1a1ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:15.145919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:15.147151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:15.148193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:15.153835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 401/794] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:15.159661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cef204-4499-47c3-8d16-6241a0c1a1ec None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:15.195718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:15.200444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:15.245222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd4ca1-d2ad-4f93-b9f4-de54dddf68b0 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:15.300102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539a6ec-2362-4f21-8f15-9dbd1061990f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:15.301255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539a6ec-2362-4f21-8f15-9dbd1061990f None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:15.301755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539a6ec-2362-4f21-8f15-9dbd1061990f None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:15.302246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539a6ec-2362-4f21-8f15-9dbd1061990f None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:15.303288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539a6ec-2362-4f21-8f15-9dbd1061990f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:15.303814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539a6ec-2362-4f21-8f15-9dbd1061990f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:15.305497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539a6ec-2362-4f21-8f15-9dbd1061990f None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:15.333217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cef204-4499-47c3-8d16-6241a0c1a1ec None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:15.334410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 394/795] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:15.339898 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539a6ec-2362-4f21-8f15-9dbd1061990f None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:15.370334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e829f2-b185-4c3b-8bd1-19d86d6d2d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:15.371035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e829f2-b185-4c3b-8bd1-19d86d6d2d58 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:15.371315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e829f2-b185-4c3b-8bd1-19d86d6d2d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:15.371587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e829f2-b185-4c3b-8bd1-19d86d6d2d58 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:15.372335 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e829f2-b185-4c3b-8bd1-19d86d6d2d58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:15.372644 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e829f2-b185-4c3b-8bd1-19d86d6d2d58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:15.374048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e829f2-b185-4c3b-8bd1-19d86d6d2d58 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:15.383246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 402/796] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:13:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:15.395804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e829f2-b185-4c3b-8bd1-19d86d6d2d58 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:15.473949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539a6ec-2362-4f21-8f15-9dbd1061990f None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:15.486754 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:15.501279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:15.512204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e829f2-b185-4c3b-8bd1-19d86d6d2d58 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:15.513464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe43e5-8650-49f6-bf83-360af8018e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:15.514645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe43e5-8650-49f6-bf83-360af8018e3d None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:15.515147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe43e5-8650-49f6-bf83-360af8018e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:15.515655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe43e5-8650-49f6-bf83-360af8018e3d None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/0603019b08d94980863f6612907601a9/roles/cb4aa5570b15465eb4e81979b1019570` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:15.531698 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe43e5-8650-49f6-bf83-360af8018e3d None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=0603019b08d94980863f6612907601a9, role_id=cb4aa5570b15465eb4e81979b1019570)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:15.533688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe43e5-8650-49f6-bf83-360af8018e3d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:15.550440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7485de-b6d2-4735-a968-ea579022ac6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:15.551507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7485de-b6d2-4735-a968-ea579022ac6a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:15.551940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7485de-b6d2-4735-a968-ea579022ac6a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:15.552271 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7485de-b6d2-4735-a968-ea579022ac6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:15.559121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 395/797] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:13:15 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/0603019b08d94980863f6612907601a9/roles/cb4aa5570b15465eb4e81979b1019570 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:13:15.565617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7485de-b6d2-4735-a968-ea579022ac6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:15.567199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7485de-b6d2-4735-a968-ea579022ac6a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ec7485de-b6d2-4735-a968-ea579022ac6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:15.587860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:15.588885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:15.589804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:15.589804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:15.589804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:15.589804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:15.589804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 403/798] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:15.595078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe43e5-8650-49f6-bf83-360af8018e3d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:15.657760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5c675-e2a4-44e6-aa57-258a106417f2 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:15.658131 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5c675-e2a4-44e6-aa57-258a106417f2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:15.658379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5c675-e2a4-44e6-aa57-258a106417f2 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:15.661297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 404/799] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:15.666192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c7362-fcf0-4bdc-95c7-6c0b45941f7d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:15.667039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c7362-fcf0-4bdc-95c7-6c0b45941f7d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:15.667329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c7362-fcf0-4bdc-95c7-6c0b45941f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:15.687490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d06c7362-fcf0-4bdc-95c7-6c0b45941f7d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:15.689066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d06c7362-fcf0-4bdc-95c7-6c0b45941f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:15.754110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe43e5-8650-49f6-bf83-360af8018e3d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:15.813959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17698f1-ab85-44ff-95ea-1181a172aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:15.814789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17698f1-ab85-44ff-95ea-1181a172aff4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:15.814932 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17698f1-ab85-44ff-95ea-1181a172aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:15.815272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17698f1-ab85-44ff-95ea-1181a172aff4 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:15.815863 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17698f1-ab85-44ff-95ea-1181a172aff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:15.818569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17698f1-ab85-44ff-95ea-1181a172aff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:15.818569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17698f1-ab85-44ff-95ea-1181a172aff4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:15.854928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 396/800] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:15.860686 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17698f1-ab85-44ff-95ea-1181a172aff4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:15.862484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 405/801] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:15.895773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c7362-fcf0-4bdc-95c7-6c0b45941f7d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:15.904192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:15.985066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:16.003424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17698f1-ab85-44ff-95ea-1181a172aff4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:16.042116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c7362-fcf0-4bdc-95c7-6c0b45941f7d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:16.044457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce70f2e-dcbb-4294-972d-651fcf0b6b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:16.045342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce70f2e-dcbb-4294-972d-651fcf0b6b48 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:16.045701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce70f2e-dcbb-4294-972d-651fcf0b6b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:16.046050 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce70f2e-dcbb-4294-972d-651fcf0b6b48 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:16.054542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce70f2e-dcbb-4294-972d-651fcf0b6b48 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:16.056192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce70f2e-dcbb-4294-972d-651fcf0b6b48 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-8ce70f2e-dcbb-4294-972d-651fcf0b6b48 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:16.071542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:16.074015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:16.075126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:16.075126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:16.075126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:16.075126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:16.075126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:16.075126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:16.075126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:16.075126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:16.075126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:13:16.075126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:16.075126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 397/802] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:16.079621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce70f2e-dcbb-4294-972d-651fcf0b6b48 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:16.081619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24278fab-ea4a-40f3-bb0c-d1fc0c30aba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:16.082488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24278fab-ea4a-40f3-bb0c-d1fc0c30aba9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:16.082861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24278fab-ea4a-40f3-bb0c-d1fc0c30aba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:16.083198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24278fab-ea4a-40f3-bb0c-d1fc0c30aba9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:16.097333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24278fab-ea4a-40f3-bb0c-d1fc0c30aba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-24278fab-ea4a-40f3-bb0c-d1fc0c30aba9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:16.106227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:16.108307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:16.109600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 406/803] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:16.115580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24278fab-ea4a-40f3-bb0c-d1fc0c30aba9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:16.229697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24278fab-ea4a-40f3-bb0c-d1fc0c30aba9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:16.254208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:16.269142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa2b8d-ba08-4764-9a5c-635d9aa58c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:16.269817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa2b8d-ba08-4764-9a5c-635d9aa58c41 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:16.270168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa2b8d-ba08-4764-9a5c-635d9aa58c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:16.270598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa2b8d-ba08-4764-9a5c-635d9aa58c41 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:16.271727 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa2b8d-ba08-4764-9a5c-635d9aa58c41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:16.272034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa2b8d-ba08-4764-9a5c-635d9aa58c41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:16.273683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa2b8d-ba08-4764-9a5c-635d9aa58c41 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:16.288922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 407/804] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:13:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:16.290779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa2b8d-ba08-4764-9a5c-635d9aa58c41 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:16.318911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:16.332292 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce70f2e-dcbb-4294-972d-651fcf0b6b48 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:16.406669 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75b007-cdd0-4e25-8923-8dc85438de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:16.407349 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75b007-cdd0-4e25-8923-8dc85438de20 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:16.407626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75b007-cdd0-4e25-8923-8dc85438de20 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:16.407897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75b007-cdd0-4e25-8923-8dc85438de20 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:16.408742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75b007-cdd0-4e25-8923-8dc85438de20 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:16.409050 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75b007-cdd0-4e25-8923-8dc85438de20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:16.410501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75b007-cdd0-4e25-8923-8dc85438de20 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:16.444500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa2b8d-ba08-4764-9a5c-635d9aa58c41 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:16.452229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 398/805] 10.222.0.93 () {56 vars in 1135 bytes} [Fri Jun 3 15:13:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:16.457366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75b007-cdd0-4e25-8923-8dc85438de20 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:16.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd767e5-d4fe-49cd-82e8-84a49c768356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:16.507298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd767e5-d4fe-49cd-82e8-84a49c768356 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:16.507565 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd767e5-d4fe-49cd-82e8-84a49c768356 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:16.507838 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd767e5-d4fe-49cd-82e8-84a49c768356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:16.543365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd767e5-d4fe-49cd-82e8-84a49c768356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:16.544854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd767e5-d4fe-49cd-82e8-84a49c768356 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-7cd767e5-d4fe-49cd-82e8-84a49c768356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:16.583840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:16.586462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:16.589436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:16.589436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:16.589436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:16.589436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:16.594717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 408/806] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:16.601315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd767e5-d4fe-49cd-82e8-84a49c768356 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:16.646244 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75b007-cdd0-4e25-8923-8dc85438de20 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:16.680858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:16.692026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fa379-6539-4506-bb93-97732ba26ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:16.692925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fa379-6539-4506-bb93-97732ba26ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:16.693207 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fa379-6539-4506-bb93-97732ba26ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:16.693475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fa379-6539-4506-bb93-97732ba26ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:16.707544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fa379-6539-4506-bb93-97732ba26ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:16.709023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16fa379-6539-4506-bb93-97732ba26ea3 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-c16fa379-6539-4506-bb93-97732ba26ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:16.730662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:16.732832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:16.734775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:16.734775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:16.734775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:16.734775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:16.734775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 399/807] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:16.743530 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fa379-6539-4506-bb93-97732ba26ea3 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:16.748240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:16.781723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd767e5-d4fe-49cd-82e8-84a49c768356 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:16.819247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100545f-b455-4b7c-8881-3ee40ccc7da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:16.819918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100545f-b455-4b7c-8881-3ee40ccc7da2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:16.820200 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100545f-b455-4b7c-8881-3ee40ccc7da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:16.820473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100545f-b455-4b7c-8881-3ee40ccc7da2 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:16.821206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100545f-b455-4b7c-8881-3ee40ccc7da2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:16.821508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100545f-b455-4b7c-8881-3ee40ccc7da2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:16.823543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100545f-b455-4b7c-8881-3ee40ccc7da2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:16.836854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 409/808] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:16.843362 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100545f-b455-4b7c-8881-3ee40ccc7da2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:16.933354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16fa379-6539-4506-bb93-97732ba26ea3 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:16.981912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f265ec5-2d01-4a1b-9fc3-38f169419f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:16.983034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265ec5-2d01-4a1b-9fc3-38f169419f92 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:16.983542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265ec5-2d01-4a1b-9fc3-38f169419f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:16.984042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f265ec5-2d01-4a1b-9fc3-38f169419f92 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:16.985019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f265ec5-2d01-4a1b-9fc3-38f169419f92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:16.985530 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f265ec5-2d01-4a1b-9fc3-38f169419f92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:16.987184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f265ec5-2d01-4a1b-9fc3-38f169419f92 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:17.006551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 400/809] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:17.018557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f265ec5-2d01-4a1b-9fc3-38f169419f92 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:17.086964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:17.087614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:17.098957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100545f-b455-4b7c-8881-3ee40ccc7da2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:17.154221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4b991-5088-4cad-8d04-e2dd1a33f6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:17.155079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4b991-5088-4cad-8d04-e2dd1a33f6cd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.155360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4b991-5088-4cad-8d04-e2dd1a33f6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.155629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4b991-5088-4cad-8d04-e2dd1a33f6cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.158272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f265ec5-2d01-4a1b-9fc3-38f169419f92 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:17.164777 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4b991-5088-4cad-8d04-e2dd1a33f6cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:17.166267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4b991-5088-4cad-8d04-e2dd1a33f6cd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-3ec4b991-5088-4cad-8d04-e2dd1a33f6cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:17.187608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:17.188679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:17.189638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:17.189638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:17.189638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:17.189638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:17.189638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:17.189638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:17.189638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:17.189638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:17.189638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:13:17.189638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:17.189638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 410/810] 10.222.0.93 () {56 vars in 1093 bytes} [Fri Jun 3 15:13:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:17.195329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082e699-0533-4713-b90b-63ba43e2a41c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.195904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082e699-0533-4713-b90b-63ba43e2a41c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.196222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082e699-0533-4713-b90b-63ba43e2a41c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.198359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 411/811] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:17.199607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082e699-0533-4713-b90b-63ba43e2a41c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:17.203813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a0ee7-9d04-4fa0-b001-79af89680790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:17.204715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a0ee7-9d04-4fa0-b001-79af89680790 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.205072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a0ee7-9d04-4fa0-b001-79af89680790 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.205429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a0ee7-9d04-4fa0-b001-79af89680790 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.213796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a0ee7-9d04-4fa0-b001-79af89680790 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:17.215485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a0ee7-9d04-4fa0-b001-79af89680790 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-e71a0ee7-9d04-4fa0-b001-79af89680790 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:17.233551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:17.235565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:17.237383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:17.237383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:17.237383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:17.237383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:17.237383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:17.237383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:17.237383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:17.237383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:17.237383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 401/812] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:17.242488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74022cc6-d97d-47f4-b0f7-26bfc0d8c7f4 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.242488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74022cc6-d97d-47f4-b0f7-26bfc0d8c7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.242488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74022cc6-d97d-47f4-b0f7-26bfc0d8c7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.261598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-74022cc6-d97d-47f4-b0f7-26bfc0d8c7f4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:17.263073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-74022cc6-d97d-47f4-b0f7-26bfc0d8c7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:17.328007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082e699-0533-4713-b90b-63ba43e2a41c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:17.350978 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 402/813] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:17.351986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74022cc6-d97d-47f4-b0f7-26bfc0d8c7f4 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:17.364943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e61e5c-9d74-425d-91e3-a9fad0a78176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:17.365685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e61e5c-9d74-425d-91e3-a9fad0a78176 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.365972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e61e5c-9d74-425d-91e3-a9fad0a78176 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.366244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e61e5c-9d74-425d-91e3-a9fad0a78176 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.367267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e61e5c-9d74-425d-91e3-a9fad0a78176 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:17.367580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e61e5c-9d74-425d-91e3-a9fad0a78176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:17.369176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e61e5c-9d74-425d-91e3-a9fad0a78176 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:17.400326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 412/814] 10.222.0.93 () {56 vars in 1133 bytes} [Fri Jun 3 15:13:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:17.401279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e61e5c-9d74-425d-91e3-a9fad0a78176 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:17.417945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:17.431358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:17.484904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74022cc6-d97d-47f4-b0f7-26bfc0d8c7f4 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:17.508178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e61e5c-9d74-425d-91e3-a9fad0a78176 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:17.524345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46aa37-13df-4ae9-bc3f-db6a0ceb775e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:17.525023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46aa37-13df-4ae9-bc3f-db6a0ceb775e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.525310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46aa37-13df-4ae9-bc3f-db6a0ceb775e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.525711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46aa37-13df-4ae9-bc3f-db6a0ceb775e None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.527048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46aa37-13df-4ae9-bc3f-db6a0ceb775e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:17.527360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46aa37-13df-4ae9-bc3f-db6a0ceb775e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:17.528932 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46aa37-13df-4ae9-bc3f-db6a0ceb775e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:17.542743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 403/815] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:17.543639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46aa37-13df-4ae9-bc3f-db6a0ceb775e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:17.550983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f917d-037b-44ce-971d-b6c860ed88d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:17.551797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f917d-037b-44ce-971d-b6c860ed88d7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.552083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f917d-037b-44ce-971d-b6c860ed88d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.552365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f917d-037b-44ce-971d-b6c860ed88d7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.553028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f917d-037b-44ce-971d-b6c860ed88d7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:17.554518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f917d-037b-44ce-971d-b6c860ed88d7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bd0f917d-037b-44ce-971d-b6c860ed88d7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:17.566766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:17.567913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 03 15:13:17.568955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:17.573232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 413/816] 10.222.0.93 () {56 vars in 1129 bytes} [Fri Jun 3 15:13:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:17.574644 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f3303-8521-45a9-9529-b9c16d0ead7b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.574937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f3303-8521-45a9-9529-b9c16d0ead7b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.575216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f3303-8521-45a9-9529-b9c16d0ead7b None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.576522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 414/817] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:17.577190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f3303-8521-45a9-9529-b9c16d0ead7b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:17.661496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46aa37-13df-4ae9-bc3f-db6a0ceb775e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:17.662205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:17.682288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:17.690652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f3303-8521-45a9-9529-b9c16d0ead7b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:17.709774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb94f0e-ba38-4d8b-b8e9-a4388748825f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:17.711044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb94f0e-ba38-4d8b-b8e9-a4388748825f None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.711561 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb94f0e-ba38-4d8b-b8e9-a4388748825f None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.712073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb94f0e-ba38-4d8b-b8e9-a4388748825f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.728871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460157a-a314-4699-bbc8-b6ce909dda8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:17.730924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460157a-a314-4699-bbc8-b6ce909dda8d None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.731261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460157a-a314-4699-bbc8-b6ce909dda8d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.731544 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9460157a-a314-4699-bbc8-b6ce909dda8d None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/bb4fb8f859dc4489b8c4fd8fdc09c3f6/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.734008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460157a-a314-4699-bbc8-b6ce909dda8d None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=bb4fb8f859dc4489b8c4fd8fdc09c3f6, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:17.735664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9460157a-a314-4699-bbc8-b6ce909dda8d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:17.755737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 415/818] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:13:17 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/bb4fb8f859dc4489b8c4fd8fdc09c3f6/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:13:17.756930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460157a-a314-4699-bbc8-b6ce909dda8d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:17.813039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb94f0e-ba38-4d8b-b8e9-a4388748825f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:17.815140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb94f0e-ba38-4d8b-b8e9-a4388748825f None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:17.887334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9460157a-a314-4699-bbc8-b6ce909dda8d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:17.937799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41b781-9b02-470d-9f4b-44200b42ed0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:17.938401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41b781-9b02-470d-9f4b-44200b42ed0f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.938734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41b781-9b02-470d-9f4b-44200b42ed0f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.939014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41b781-9b02-470d-9f4b-44200b42ed0f None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.939888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41b781-9b02-470d-9f4b-44200b42ed0f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:17.940189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41b781-9b02-470d-9f4b-44200b42ed0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:17.941598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41b781-9b02-470d-9f4b-44200b42ed0f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:17.949154 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 416/819] 10.222.0.93 () {56 vars in 1124 bytes} [Fri Jun 3 15:13:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:17.950591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae1305-6be0-464b-91d2-d38b794d6e59 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:17.950878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae1305-6be0-464b-91d2-d38b794d6e59 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:17.951161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae1305-6be0-464b-91d2-d38b794d6e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:17.957768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:17.972078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 404/820] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:13:17.973842 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb94f0e-ba38-4d8b-b8e9-a4388748825f None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:17.984197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1fae1305-6be0-464b-91d2-d38b794d6e59 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:17.986024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1fae1305-6be0-464b-91d2-d38b794d6e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:18.002412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:18.085184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 417/821] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:18.086849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae1305-6be0-464b-91d2-d38b794d6e59 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:18.124910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb94f0e-ba38-4d8b-b8e9-a4388748825f None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:18.168103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5393f10-7fa0-4473-82b0-d063d92aa038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:18.169174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393f10-7fa0-4473-82b0-d063d92aa038 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:18.169603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393f10-7fa0-4473-82b0-d063d92aa038 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:18.170848 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5393f10-7fa0-4473-82b0-d063d92aa038 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:18.171721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5393f10-7fa0-4473-82b0-d063d92aa038 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:18.172254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5393f10-7fa0-4473-82b0-d063d92aa038 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:18.173783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5393f10-7fa0-4473-82b0-d063d92aa038 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:18.181025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 405/822] 10.222.0.93 () {56 vars in 1124 bytes} [Fri Jun 3 15:13:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:18.182446 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5393f10-7fa0-4473-82b0-d063d92aa038 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:18.203224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae1305-6be0-464b-91d2-d38b794d6e59 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:18.240500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:18.247753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ec971-8c6c-4f16-be11-7e77b1fcaef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:18.248759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ec971-8c6c-4f16-be11-7e77b1fcaef1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:18.249208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ec971-8c6c-4f16-be11-7e77b1fcaef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:18.249622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ec971-8c6c-4f16-be11-7e77b1fcaef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:18.261239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ec971-8c6c-4f16-be11-7e77b1fcaef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:18.262910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ec971-8c6c-4f16-be11-7e77b1fcaef1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:18.277520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-108ec971-8c6c-4f16-be11-7e77b1fcaef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:18.283244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:18.287928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:18.288845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:18.288845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:18.288845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:18.288845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:18.290706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 418/823] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:18.290706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ec971-8c6c-4f16-be11-7e77b1fcaef1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:18.302076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5393f10-7fa0-4473-82b0-d063d92aa038 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:18.339268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49d69a-2263-4151-b7dd-d9a46fd0e58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:18.340143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49d69a-2263-4151-b7dd-d9a46fd0e58f None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:18.340434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49d69a-2263-4151-b7dd-d9a46fd0e58f None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:18.340712 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49d69a-2263-4151-b7dd-d9a46fd0e58f None admin] PATH_INFO: `/v3/services/network` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:18.341379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49d69a-2263-4151-b7dd-d9a46fd0e58f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:18.342834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49d69a-2263-4151-b7dd-d9a46fd0e58f None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ab49d69a-2263-4151-b7dd-d9a46fd0e58f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:18.355719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:18.358271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:18.360398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 406/824] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:18 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:18.363606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49d69a-2263-4151-b7dd-d9a46fd0e58f None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:18.412190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ec971-8c6c-4f16-be11-7e77b1fcaef1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:18.449795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4b65b-dc48-47a6-86ce-fd683dcaf3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:18.450603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b65b-dc48-47a6-86ce-fd683dcaf3d5 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:18.450890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b65b-dc48-47a6-86ce-fd683dcaf3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:18.451167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4b65b-dc48-47a6-86ce-fd683dcaf3d5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:18.452124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4b65b-dc48-47a6-86ce-fd683dcaf3d5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:18.453349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e4b65b-dc48-47a6-86ce-fd683dcaf3d5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:18.474441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 419/825] 10.222.0.93 () {60 vars in 1144 bytes} [Fri Jun 3 15:13:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:18.477225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76ba46-d63b-4432-a93d-fc2aadf78159 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:18.477225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76ba46-d63b-4432-a93d-fc2aadf78159 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:18.477225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76ba46-d63b-4432-a93d-fc2aadf78159 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:18.478191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 420/826] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:18.479478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76ba46-d63b-4432-a93d-fc2aadf78159 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:18.486585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49d69a-2263-4151-b7dd-d9a46fd0e58f None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:18.538296 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed679246-707d-49f8-b72a-40d7d6a396d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:18.539070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed679246-707d-49f8-b72a-40d7d6a396d4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:18.539367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed679246-707d-49f8-b72a-40d7d6a396d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:18.539663 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed679246-707d-49f8-b72a-40d7d6a396d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:18.540702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed679246-707d-49f8-b72a-40d7d6a396d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:18.540832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed679246-707d-49f8-b72a-40d7d6a396d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:18.542920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed679246-707d-49f8-b72a-40d7d6a396d4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:18.551153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:18.554603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:18.569779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 407/827] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:18.569779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575e8ef-b752-46e5-b013-c29ba2c43434 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:18.569779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575e8ef-b752-46e5-b013-c29ba2c43434 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:18.569779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575e8ef-b752-46e5-b013-c29ba2c43434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:18.600273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2575e8ef-b752-46e5-b013-c29ba2c43434 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:18.601898 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2575e8ef-b752-46e5-b013-c29ba2c43434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:18.636086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76ba46-d63b-4432-a93d-fc2aadf78159 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:18.683294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b55c20-de65-4fc9-aa96-3e057ff5d506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:18.684032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b55c20-de65-4fc9-aa96-3e057ff5d506 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:18.684227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b55c20-de65-4fc9-aa96-3e057ff5d506 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:18.684507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b55c20-de65-4fc9-aa96-3e057ff5d506 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:18.685279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b55c20-de65-4fc9-aa96-3e057ff5d506 None admin] RBAC: Adding query filter params (name=network) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:18.685656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b55c20-de65-4fc9-aa96-3e057ff5d506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:18.687113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b55c20-de65-4fc9-aa96-3e057ff5d506 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:18.693982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 421/828] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:18.694866 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b55c20-de65-4fc9-aa96-3e057ff5d506 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:18.702013 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 408/829] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:18.703221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575e8ef-b752-46e5-b013-c29ba2c43434 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:18.822156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b55c20-de65-4fc9-aa96-3e057ff5d506 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:18.825554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:18.847545 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575e8ef-b752-46e5-b013-c29ba2c43434 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:18.866609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c0961-74f0-44cb-a0b5-9b7f549838d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:18.867551 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c0961-74f0-44cb-a0b5-9b7f549838d0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:18.867926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c0961-74f0-44cb-a0b5-9b7f549838d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:18.868322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c0961-74f0-44cb-a0b5-9b7f549838d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:18.875397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c0961-74f0-44cb-a0b5-9b7f549838d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:18.876943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c0961-74f0-44cb-a0b5-9b7f549838d0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:18.885313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-0c9c0961-74f0-44cb-a0b5-9b7f549838d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:18.892238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:18.893330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:18.894436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:18.894436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:18.894436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:18.894436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:18.894436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:18.894436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:18.894436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:18.894436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:18.902343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 422/830] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:18.902343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c0961-74f0-44cb-a0b5-9b7f549838d0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:18.905291 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03db59-31cd-4d81-818c-54103b07d98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:18.905970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03db59-31cd-4d81-818c-54103b07d98b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:18.906251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03db59-31cd-4d81-818c-54103b07d98b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:18.906524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03db59-31cd-4d81-818c-54103b07d98b None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:18.907286 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03db59-31cd-4d81-818c-54103b07d98b None admin] RBAC: Adding query filter params (type=network) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:18.909990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03db59-31cd-4d81-818c-54103b07d98b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:18.911697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d03db59-31cd-4d81-818c-54103b07d98b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:18.921354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 409/831] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:18.922858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03db59-31cd-4d81-818c-54103b07d98b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:19.029472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c0961-74f0-44cb-a0b5-9b7f549838d0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:19.052553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03db59-31cd-4d81-818c-54103b07d98b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:19.070465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef9b1f-ca67-42ba-9e4c-6f1486d011ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:19.071701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef9b1f-ca67-42ba-9e4c-6f1486d011ce None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:19.072260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef9b1f-ca67-42ba-9e4c-6f1486d011ce None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:19.072764 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef9b1f-ca67-42ba-9e4c-6f1486d011ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:19.084361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef9b1f-ca67-42ba-9e4c-6f1486d011ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-1aef9b1f-ca67-42ba-9e4c-6f1486d011ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:19.093888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:19.094884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:13:19.095758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:19.096407 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127c0b5-5ff3-4ece-9fe5-ec83792bc293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:19.096407 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127c0b5-5ff3-4ece-9fe5-ec83792bc293 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:19.096407 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127c0b5-5ff3-4ece-9fe5-ec83792bc293 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:19.096407 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127c0b5-5ff3-4ece-9fe5-ec83792bc293 None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:19.096701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127c0b5-5ff3-4ece-9fe5-ec83792bc293 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:19.096701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127c0b5-5ff3-4ece-9fe5-ec83792bc293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:19.097980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127c0b5-5ff3-4ece-9fe5-ec83792bc293 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:19.099578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 423/832] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:13:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:19.100611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef9b1f-ca67-42ba-9e4c-6f1486d011ce None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:19.114112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 410/833] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:19.115452 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127c0b5-5ff3-4ece-9fe5-ec83792bc293 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:19.133717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:19.181026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:19.234133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef9b1f-ca67-42ba-9e4c-6f1486d011ce None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:19.235307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127c0b5-5ff3-4ece-9fe5-ec83792bc293 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:19.273101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b875e-2389-4209-a16e-a933b10af027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:19.274232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b875e-2389-4209-a16e-a933b10af027 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:19.274713 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b875e-2389-4209-a16e-a933b10af027 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:19.275150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b875e-2389-4209-a16e-a933b10af027 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:19.276737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4990fc3-837c-4397-8df9-227c402ea523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:19.276931 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b875e-2389-4209-a16e-a933b10af027 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:19.277372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b875e-2389-4209-a16e-a933b10af027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:19.277677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4990fc3-837c-4397-8df9-227c402ea523 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:19.278074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4990fc3-837c-4397-8df9-227c402ea523 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:19.278451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4990fc3-837c-4397-8df9-227c402ea523 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:19.279327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4990fc3-837c-4397-8df9-227c402ea523 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:19.279510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b875e-2389-4209-a16e-a933b10af027 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:19.280944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4990fc3-837c-4397-8df9-227c402ea523 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:19.289643 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 411/834] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:13:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:19.290928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b875e-2389-4209-a16e-a933b10af027 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:19.306619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 424/835] 10.222.0.93 () {60 vars in 1144 bytes} [Fri Jun 3 15:13:19 2022] POST /identity/v3/endpoints => generated 327 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:19.308092 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4990fc3-837c-4397-8df9-227c402ea523 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:19.494628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:19.522564 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b875e-2389-4209-a16e-a933b10af027 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:19.540080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4990fc3-837c-4397-8df9-227c402ea523 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:19.540966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:19.574077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3124186-9ca4-47c0-a4d4-52c7981a94a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:19.574787 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3124186-9ca4-47c0-a4d4-52c7981a94a2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:19.575040 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3124186-9ca4-47c0-a4d4-52c7981a94a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:19.575315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3124186-9ca4-47c0-a4d4-52c7981a94a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:19.576158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3124186-9ca4-47c0-a4d4-52c7981a94a2 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=23546323ca33412fb388afe308a97499, user.id=c1966524091242e7a6cd2758027a0d4f) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:19.576469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3124186-9ca4-47c0-a4d4-52c7981a94a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:19.578092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3124186-9ca4-47c0-a4d4-52c7981a94a2 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:19.582956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57015cad-d588-499a-b99b-983a288ffb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:19.584014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57015cad-d588-499a-b99b-983a288ffb9e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:19.584503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57015cad-d588-499a-b99b-983a288ffb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:19.584990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57015cad-d588-499a-b99b-983a288ffb9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:19.590673 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 412/836] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:19 2022] GET /identity/v3/role_assignments?user.id=c1966524091242e7a6cd2758027a0d4f&scope.project.id=23546323ca33412fb388afe308a97499&role.id=d67707782358417ba9f25b2793ca2890 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:19.599006 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57015cad-d588-499a-b99b-983a288ffb9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:19.600902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57015cad-d588-499a-b99b-983a288ffb9e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-57015cad-d588-499a-b99b-983a288ffb9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:19.621458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:19.622436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:19.623401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:19.623401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:19.623401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:19.623401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:19.625776 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 425/837] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:19.631153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3124186-9ca4-47c0-a4d4-52c7981a94a2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:19.775276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3124186-9ca4-47c0-a4d4-52c7981a94a2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:19.821406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a513c-60c5-4ed5-87e1-c8c37560b2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:19.822086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a513c-60c5-4ed5-87e1-c8c37560b2aa None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:19.822367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a513c-60c5-4ed5-87e1-c8c37560b2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:19.822707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a513c-60c5-4ed5-87e1-c8c37560b2aa None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:19.823444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a513c-60c5-4ed5-87e1-c8c37560b2aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:19.823755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a513c-60c5-4ed5-87e1-c8c37560b2aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:19.825146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a513c-60c5-4ed5-87e1-c8c37560b2aa None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:19.837878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 413/838] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:19.843839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57015cad-d588-499a-b99b-983a288ffb9e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:19.959055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57015cad-d588-499a-b99b-983a288ffb9e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:19.988138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:19.997464 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb7b2c-e933-410e-b6c1-8e3e9fb16753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:19.998567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb7b2c-e933-410e-b6c1-8e3e9fb16753 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:19.999036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb7b2c-e933-410e-b6c1-8e3e9fb16753 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:19.999493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb7b2c-e933-410e-b6c1-8e3e9fb16753 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:20.008274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb7b2c-e933-410e-b6c1-8e3e9fb16753 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:20.009953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb7b2c-e933-410e-b6c1-8e3e9fb16753 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e7fb7b2c-e933-410e-b6c1-8e3e9fb16753 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:20.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:20.027717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:20.028969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:20.028969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:20.028969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:20.028969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:20.028969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:20.028969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:20.028969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:20.028969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:20.028969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:20.028969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:20.031297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 426/839] 10.222.0.93 () {56 vars in 1117 bytes} [Fri Jun 3 15:13:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:20.037192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a513c-60c5-4ed5-87e1-c8c37560b2aa None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:20.079146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:20.144663 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a513c-60c5-4ed5-87e1-c8c37560b2aa None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:20.180802 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914b1a6-7806-4e24-8381-f8d8cc879957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:20.181469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914b1a6-7806-4e24-8381-f8d8cc879957 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:20.181748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914b1a6-7806-4e24-8381-f8d8cc879957 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:20.182051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914b1a6-7806-4e24-8381-f8d8cc879957 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:20.182917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914b1a6-7806-4e24-8381-f8d8cc879957 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:20.183223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914b1a6-7806-4e24-8381-f8d8cc879957 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:20.184661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8914b1a6-7806-4e24-8381-f8d8cc879957 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:20.203553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 414/840] 10.222.0.93 () {56 vars in 1149 bytes} [Fri Jun 3 15:13:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:20.208824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb7b2c-e933-410e-b6c1-8e3e9fb16753 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:20.300960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109a559-9301-472e-b483-5fadaccdef1e None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:20.301157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109a559-9301-472e-b483-5fadaccdef1e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:20.301238 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109a559-9301-472e-b483-5fadaccdef1e None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:20.302852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 415/841] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:20.308737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb66c1-1410-4159-9426-923f11dffe5f None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:20.309020 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb66c1-1410-4159-9426-923f11dffe5f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:20.309295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eb66c1-1410-4159-9426-923f11dffe5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:20.319796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb7b2c-e933-410e-b6c1-8e3e9fb16753 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:20.330114 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-39eb66c1-1410-4159-9426-923f11dffe5f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:20.331720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-39eb66c1-1410-4159-9426-923f11dffe5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:20.360055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f8c1-03b4-4f5c-9fc1-c544526b1bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:20.361041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f8c1-03b4-4f5c-9fc1-c544526b1bcb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:20.361564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f8c1-03b4-4f5c-9fc1-c544526b1bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:20.362117 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f0f8c1-03b4-4f5c-9fc1-c544526b1bcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:20.426617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 416/842] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:20.431882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:20.448206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0f8c1-03b4-4f5c-9fc1-c544526b1bcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:20.450102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f0f8c1-03b4-4f5c-9fc1-c544526b1bcb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:20.451092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb66c1-1410-4159-9426-923f11dffe5f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:20.486172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:20.572354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 427/843] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:20 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:13:20.578403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f8c1-03b4-4f5c-9fc1-c544526b1bcb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:20.580957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eb66c1-1410-4159-9426-923f11dffe5f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:20.630044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5f393-dd69-4399-a661-2ce39dbae64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:20.630884 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5f393-dd69-4399-a661-2ce39dbae64c None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:20.631162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5f393-dd69-4399-a661-2ce39dbae64c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:20.631433 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5f393-dd69-4399-a661-2ce39dbae64c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:20.644318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f5f393-dd69-4399-a661-2ce39dbae64c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-79f5f393-dd69-4399-a661-2ce39dbae64c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:20.658190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:20.659225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:20.660115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:20.662821 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 417/844] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:20.665585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5f393-dd69-4399-a661-2ce39dbae64c None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:20.706980 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:20.724641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f0f8c1-03b4-4f5c-9fc1-c544526b1bcb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:20.783185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a405d3a-8291-4ab9-8645-03cfcb43c795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:20.784958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a405d3a-8291-4ab9-8645-03cfcb43c795 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:20.784958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a405d3a-8291-4ab9-8645-03cfcb43c795 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:20.784958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a405d3a-8291-4ab9-8645-03cfcb43c795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:20.800791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:20.809346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a405d3a-8291-4ab9-8645-03cfcb43c795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:20.811468 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a405d3a-8291-4ab9-8645-03cfcb43c795 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:20.834248 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5f393-dd69-4399-a661-2ce39dbae64c None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9a405d3a-8291-4ab9-8645-03cfcb43c795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:20.836503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:20.837401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:20.839036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:20.839036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:20.839036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:20.839036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:20.841370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 428/845] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:20.848505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a405d3a-8291-4ab9-8645-03cfcb43c795 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:20.885711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4d81c-31ae-4df3-9135-dd129b3f8e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:20.886401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4d81c-31ae-4df3-9135-dd129b3f8e73 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:20.886722 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4d81c-31ae-4df3-9135-dd129b3f8e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:20.887001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4d81c-31ae-4df3-9135-dd129b3f8e73 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:20.887766 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4d81c-31ae-4df3-9135-dd129b3f8e73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:20.888073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4d81c-31ae-4df3-9135-dd129b3f8e73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:20.889796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4d81c-31ae-4df3-9135-dd129b3f8e73 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:20.910116 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 418/846] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:13:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:20.911410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4d81c-31ae-4df3-9135-dd129b3f8e73 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:21.001061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a405d3a-8291-4ab9-8645-03cfcb43c795 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:21.042046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd764bb-e779-4ff1-8527-c1ffe6765a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:21.042759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd764bb-e779-4ff1-8527-c1ffe6765a52 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:21.043042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd764bb-e779-4ff1-8527-c1ffe6765a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:21.043326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd764bb-e779-4ff1-8527-c1ffe6765a52 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:21.044060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd764bb-e779-4ff1-8527-c1ffe6765a52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:21.044365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd764bb-e779-4ff1-8527-c1ffe6765a52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:21.045288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4d81c-31ae-4df3-9135-dd129b3f8e73 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:21.045889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd764bb-e779-4ff1-8527-c1ffe6765a52 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:21.061209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:21.070614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 429/847] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:21.070614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a966114-d167-456e-bb12-0302fc01a1aa None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:21.070614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a966114-d167-456e-bb12-0302fc01a1aa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:21.070614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a966114-d167-456e-bb12-0302fc01a1aa None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:21.070614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 430/848] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:21.075247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a966114-d167-456e-bb12-0302fc01a1aa None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:21.096152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ddf1ce-e8da-44de-9d19-0d665a8da6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:21.097479 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddf1ce-e8da-44de-9d19-0d665a8da6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:21.097944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddf1ce-e8da-44de-9d19-0d665a8da6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:21.098410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ddf1ce-e8da-44de-9d19-0d665a8da6a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:21.112929 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddf1ce-e8da-44de-9d19-0d665a8da6a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:21.114976 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ddf1ce-e8da-44de-9d19-0d665a8da6a6 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:21.125782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-07ddf1ce-e8da-44de-9d19-0d665a8da6a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:21.138283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:21.140266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:21.142203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:21.142203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:21.142203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:21.142203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:21.146561 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 419/849] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:21.147943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a735bf-9fde-4af6-84f3-7485dbdd4945 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:21.148226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a735bf-9fde-4af6-84f3-7485dbdd4945 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:21.148504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a735bf-9fde-4af6-84f3-7485dbdd4945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:21.166738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e0a735bf-9fde-4af6-84f3-7485dbdd4945 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:21.167920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e0a735bf-9fde-4af6-84f3-7485dbdd4945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:21.223793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a966114-d167-456e-bb12-0302fc01a1aa None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:21.257824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 420/850] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:21.258672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a735bf-9fde-4af6-84f3-7485dbdd4945 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:21.284128 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6049bc-90b8-4d51-a2eb-e3532ede4eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:21.285242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6049bc-90b8-4d51-a2eb-e3532ede4eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:21.285697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6049bc-90b8-4d51-a2eb-e3532ede4eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:21.286195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6049bc-90b8-4d51-a2eb-e3532ede4eb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:21.296060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6049bc-90b8-4d51-a2eb-e3532ede4eb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:21.297816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6049bc-90b8-4d51-a2eb-e3532ede4eb7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-9c6049bc-90b8-4d51-a2eb-e3532ede4eb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:21.313988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:21.315142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:21.316678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:21.316678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:21.316678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:21.316678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:21.316678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:21.316678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:21.316678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:21.316678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:21.316678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 431/851] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:21.316678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6049bc-90b8-4d51-a2eb-e3532ede4eb7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:21.392634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:21.403922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a735bf-9fde-4af6-84f3-7485dbdd4945 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:21.445330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:21.447294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6049bc-90b8-4d51-a2eb-e3532ede4eb7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:21.457169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861a88b-882a-4532-b643-3997e80c8e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:21.458191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861a88b-882a-4532-b643-3997e80c8e43 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:21.458191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861a88b-882a-4532-b643-3997e80c8e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:21.458371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861a88b-882a-4532-b643-3997e80c8e43 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:21.459237 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861a88b-882a-4532-b643-3997e80c8e43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:21.459544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861a88b-882a-4532-b643-3997e80c8e43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:21.461123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861a88b-882a-4532-b643-3997e80c8e43 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:21.474339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 421/852] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:21.475335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861a88b-882a-4532-b643-3997e80c8e43 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:21.499802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4f225-22cc-4543-9b41-184d8a4e3299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:21.500603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4f225-22cc-4543-9b41-184d8a4e3299 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:21.500890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4f225-22cc-4543-9b41-184d8a4e3299 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:21.501168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4f225-22cc-4543-9b41-184d8a4e3299 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:21.501837 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4f225-22cc-4543-9b41-184d8a4e3299 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:21.503250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af4f225-22cc-4543-9b41-184d8a4e3299 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-3af4f225-22cc-4543-9b41-184d8a4e3299 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:21.516032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:21.517361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:21.518206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 432/853] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:13:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:21.519060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4f225-22cc-4543-9b41-184d8a4e3299 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:21.606585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861a88b-882a-4532-b643-3997e80c8e43 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:21.649180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4f225-22cc-4543-9b41-184d8a4e3299 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:21.649361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a28964-8bdf-4baa-b392-379a85db1d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:21.649865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a28964-8bdf-4baa-b392-379a85db1d64 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:21.650149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a28964-8bdf-4baa-b392-379a85db1d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:21.650431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a28964-8bdf-4baa-b392-379a85db1d64 None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:21.651925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a28964-8bdf-4baa-b392-379a85db1d64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:21.652239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a28964-8bdf-4baa-b392-379a85db1d64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:21.653851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a28964-8bdf-4baa-b392-379a85db1d64 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:21.669496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 422/854] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:21.670482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a28964-8bdf-4baa-b392-379a85db1d64 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:21.688853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:21.704304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25012d04-b504-4b33-89d5-8533fceb5f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:21.705107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25012d04-b504-4b33-89d5-8533fceb5f9c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:21.705388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25012d04-b504-4b33-89d5-8533fceb5f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:21.705663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25012d04-b504-4b33-89d5-8533fceb5f9c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:21.717815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25012d04-b504-4b33-89d5-8533fceb5f9c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:21.719234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25012d04-b504-4b33-89d5-8533fceb5f9c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-25012d04-b504-4b33-89d5-8533fceb5f9c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:21.740680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:21.741748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:21.742774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:21.742774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:21.742774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:21.742774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:21.742774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:21.742774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:21.742774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:21.742774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:21.742774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 03 15:13:21.742774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:21.742774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 433/855] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:21.743458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25012d04-b504-4b33-89d5-8533fceb5f9c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:21.754808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:21.827694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a28964-8bdf-4baa-b392-379a85db1d64 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:21.869999 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5427e2-0363-4f4c-9922-e2c3965992a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:21.870243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25012d04-b504-4b33-89d5-8533fceb5f9c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:21.871163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5427e2-0363-4f4c-9922-e2c3965992a5 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:21.872014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5427e2-0363-4f4c-9922-e2c3965992a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:21.872318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5427e2-0363-4f4c-9922-e2c3965992a5 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:21.874184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5427e2-0363-4f4c-9922-e2c3965992a5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:21.876646 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5427e2-0363-4f4c-9922-e2c3965992a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:21.876646 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5427e2-0363-4f4c-9922-e2c3965992a5 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:21.886723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 423/856] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:13:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:21.887663 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5427e2-0363-4f4c-9922-e2c3965992a5 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:21.922597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f4a25-49bb-450f-8a51-c45ac9e12a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:21.923345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f4a25-49bb-450f-8a51-c45ac9e12a50 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:21.923713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f4a25-49bb-450f-8a51-c45ac9e12a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:21.923987 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f4a25-49bb-450f-8a51-c45ac9e12a50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:21.924830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f4a25-49bb-450f-8a51-c45ac9e12a50 None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=0603019b08d94980863f6612907601a9) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:21.925136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f4a25-49bb-450f-8a51-c45ac9e12a50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:21.926622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f4a25-49bb-450f-8a51-c45ac9e12a50 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:21.942563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 434/857] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:21 2022] GET /identity/v3/role_assignments?user.id=0603019b08d94980863f6612907601a9&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:21.942563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f4a25-49bb-450f-8a51-c45ac9e12a50 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:22.021420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:22.035226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5427e2-0363-4f4c-9922-e2c3965992a5 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:22.057353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:22.084702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359327c-db33-4ffb-ad91-749922922e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:22.085533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359327c-db33-4ffb-ad91-749922922e5d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:22.085651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359327c-db33-4ffb-ad91-749922922e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:22.085935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359327c-db33-4ffb-ad91-749922922e5d None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:22.086835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359327c-db33-4ffb-ad91-749922922e5d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:22.087147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359327c-db33-4ffb-ad91-749922922e5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:22.088656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359327c-db33-4ffb-ad91-749922922e5d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:22.097785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f4a25-49bb-450f-8a51-c45ac9e12a50 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:22.114882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 424/858] 10.222.0.93 () {56 vars in 1135 bytes} [Fri Jun 3 15:13:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:22.114882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36f36d-5771-4fda-8110-d8ce071a004b None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:22.114882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36f36d-5771-4fda-8110-d8ce071a004b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:22.114882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c36f36d-5771-4fda-8110-d8ce071a004b None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:22.115668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 425/859] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:22.117079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36f36d-5771-4fda-8110-d8ce071a004b None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:22.142424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa297e29-d5e5-4034-8602-1e3c3ba49ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:22.143126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa297e29-d5e5-4034-8602-1e3c3ba49ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:22.143409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa297e29-d5e5-4034-8602-1e3c3ba49ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:22.143685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa297e29-d5e5-4034-8602-1e3c3ba49ea2 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:22.144408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa297e29-d5e5-4034-8602-1e3c3ba49ea2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:22.144720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa297e29-d5e5-4034-8602-1e3c3ba49ea2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:22.146108 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa297e29-d5e5-4034-8602-1e3c3ba49ea2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:22.152083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 435/860] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:13:21 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:22.153329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8f237-a875-4564-8d9e-d141fdb7a82d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:22.153329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8f237-a875-4564-8d9e-d141fdb7a82d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:22.153329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8f237-a875-4564-8d9e-d141fdb7a82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:22.173308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b7d8f237-a875-4564-8d9e-d141fdb7a82d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:22.174480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b7d8f237-a875-4564-8d9e-d141fdb7a82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:22.262972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 436/861] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:22.264129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c36f36d-5771-4fda-8110-d8ce071a004b None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:22.286946 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8f237-a875-4564-8d9e-d141fdb7a82d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:22.308832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e7600-178e-4cdc-88b6-0c805655253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:22.309504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e7600-178e-4cdc-88b6-0c805655253e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:22.309781 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e7600-178e-4cdc-88b6-0c805655253e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:22.310058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e7600-178e-4cdc-88b6-0c805655253e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:22.345728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:22.373305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:22.410450 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e7600-178e-4cdc-88b6-0c805655253e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:22.411998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e7600-178e-4cdc-88b6-0c805655253e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:22.429723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8f237-a875-4564-8d9e-d141fdb7a82d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:22.480658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d081f-8531-41af-9768-0c1926ee05ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:22.482715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d081f-8531-41af-9768-0c1926ee05ac None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:22.483205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d081f-8531-41af-9768-0c1926ee05ac None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:22.483664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d081f-8531-41af-9768-0c1926ee05ac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:22.484777 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d081f-8531-41af-9768-0c1926ee05ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:22.487663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d081f-8531-41af-9768-0c1926ee05ac None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-032d081f-8531-41af-9768-0c1926ee05ac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:22.505027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:22.506032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 15:13:22.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:22.515063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 437/862] 10.222.0.93 () {56 vars in 1132 bytes} [Fri Jun 3 15:13:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:22.516823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d081f-8531-41af-9768-0c1926ee05ac None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:22.531574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 426/863] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:22 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 15:13:22.538079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e7600-178e-4cdc-88b6-0c805655253e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:22.628934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:22.661165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d081f-8531-41af-9768-0c1926ee05ac None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:22.669960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:22.680679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e7600-178e-4cdc-88b6-0c805655253e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:22.705018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f7d40-e919-4ac4-b7cd-267ca6d52d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:22.705719 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f7d40-e919-4ac4-b7cd-267ca6d52d14 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:22.705998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f7d40-e919-4ac4-b7cd-267ca6d52d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:22.706269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f7d40-e919-4ac4-b7cd-267ca6d52d14 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:22.707297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f7d40-e919-4ac4-b7cd-267ca6d52d14 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:22.707606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f7d40-e919-4ac4-b7cd-267ca6d52d14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:22.709643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f7d40-e919-4ac4-b7cd-267ca6d52d14 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:22.718786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 438/864] 10.222.0.93 () {56 vars in 1126 bytes} [Fri Jun 3 15:13:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:22.722871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f7d40-e919-4ac4-b7cd-267ca6d52d14 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:22.723899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e81d3a-b223-42d7-8043-e87228469768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:22.724798 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e81d3a-b223-42d7-8043-e87228469768 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:22.725151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e81d3a-b223-42d7-8043-e87228469768 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:22.725497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e81d3a-b223-42d7-8043-e87228469768 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:22.740954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e81d3a-b223-42d7-8043-e87228469768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:22.743334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e81d3a-b223-42d7-8043-e87228469768 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-d5e81d3a-b223-42d7-8043-e87228469768 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:22.766298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:22.767274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:22.768172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:22.768172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:22.768172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:22.768172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:22.770359 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 427/865] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:22.776200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e81d3a-b223-42d7-8043-e87228469768 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:22.852075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f7d40-e919-4ac4-b7cd-267ca6d52d14 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:22.902457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240ecfe-f925-4857-922d-a1c48d247fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:22.903301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240ecfe-f925-4857-922d-a1c48d247fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:22.904045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240ecfe-f925-4857-922d-a1c48d247fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:22.904338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240ecfe-f925-4857-922d-a1c48d247fc8 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:22.905207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240ecfe-f925-4857-922d-a1c48d247fc8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:22.905661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240ecfe-f925-4857-922d-a1c48d247fc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:22.907684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c240ecfe-f925-4857-922d-a1c48d247fc8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:22.911191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e81d3a-b223-42d7-8043-e87228469768 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:22.914704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 439/866] 10.222.0.93 () {56 vars in 1126 bytes} [Fri Jun 3 15:13:22 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:22.944552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:22.959153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d3e1c-4f32-46a0-b3a6-9eb48a25e522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:22.960100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d3e1c-4f32-46a0-b3a6-9eb48a25e522 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:22.960618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d3e1c-4f32-46a0-b3a6-9eb48a25e522 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:22.961058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d3e1c-4f32-46a0-b3a6-9eb48a25e522 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:22.961969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d3e1c-4f32-46a0-b3a6-9eb48a25e522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:22.962437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d3e1c-4f32-46a0-b3a6-9eb48a25e522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:22.964941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d3e1c-4f32-46a0-b3a6-9eb48a25e522 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:22.979977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 428/867] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:22.989022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240ecfe-f925-4857-922d-a1c48d247fc8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:23.043845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:23.124714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240ecfe-f925-4857-922d-a1c48d247fc8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:23.165082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63c2f7-0aff-4e8b-8ece-2111dcb760f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:23.165891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c2f7-0aff-4e8b-8ece-2111dcb760f5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:23.166221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c2f7-0aff-4e8b-8ece-2111dcb760f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:23.166502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63c2f7-0aff-4e8b-8ece-2111dcb760f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:23.174396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63c2f7-0aff-4e8b-8ece-2111dcb760f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:23.175898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63c2f7-0aff-4e8b-8ece-2111dcb760f5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-5a63c2f7-0aff-4e8b-8ece-2111dcb760f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:23.192268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:23.193384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:23.194480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:23.194480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:23.194480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:23.194480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:23.194480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:23.194480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:23.194480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:23.194480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:23.194480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 440/868] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:23.200248 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d3e1c-4f32-46a0-b3a6-9eb48a25e522 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:23.314412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d3e1c-4f32-46a0-b3a6-9eb48a25e522 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:23.352730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2039a1-5904-4a97-be24-5ddf02efcf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:23.353698 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2039a1-5904-4a97-be24-5ddf02efcf1a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:23.354127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2039a1-5904-4a97-be24-5ddf02efcf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:23.354643 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2039a1-5904-4a97-be24-5ddf02efcf1a None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:23.355592 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2039a1-5904-4a97-be24-5ddf02efcf1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:23.356050 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2039a1-5904-4a97-be24-5ddf02efcf1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:23.357653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2039a1-5904-4a97-be24-5ddf02efcf1a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:23.372054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 429/869] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:23.378562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63c2f7-0aff-4e8b-8ece-2111dcb760f5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:23.497548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:23.507859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63c2f7-0aff-4e8b-8ece-2111dcb760f5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:23.554840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef72047f-8338-4090-9183-ba5ec56660bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:23.555520 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72047f-8338-4090-9183-ba5ec56660bb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:23.555803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72047f-8338-4090-9183-ba5ec56660bb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:23.556082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef72047f-8338-4090-9183-ba5ec56660bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:23.557452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72047f-8338-4090-9183-ba5ec56660bb None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=23546323ca33412fb388afe308a97499, user.id=bb4fb8f859dc4489b8c4fd8fdc09c3f6) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:23.557763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72047f-8338-4090-9183-ba5ec56660bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:23.559259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef72047f-8338-4090-9183-ba5ec56660bb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:23.566272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 441/870] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:23 2022] GET /identity/v3/role_assignments?user.id=bb4fb8f859dc4489b8c4fd8fdc09c3f6&scope.project.id=23546323ca33412fb388afe308a97499&role.id=d67707782358417ba9f25b2793ca2890 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:24.528693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54fc3-67fd-4c1f-82c4-f2a6b58ef005 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:24.529227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54fc3-67fd-4c1f-82c4-f2a6b58ef005 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:24.529227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54fc3-67fd-4c1f-82c4-f2a6b58ef005 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:24.530977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 430/871] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:24.537217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765a544-66ed-438c-987d-0a34927bfd78 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:24.538207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765a544-66ed-438c-987d-0a34927bfd78 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:24.538207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765a544-66ed-438c-987d-0a34927bfd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:24.564883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8765a544-66ed-438c-987d-0a34927bfd78 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:24.566218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8765a544-66ed-438c-987d-0a34927bfd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:24.674576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 442/872] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:24.689688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a54fc3-67fd-4c1f-82c4-f2a6b58ef005 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:24.741600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f554d7-0f48-42ef-80cc-ad4f400259ba None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:24.741600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f554d7-0f48-42ef-80cc-ad4f400259ba None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:24.741892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f554d7-0f48-42ef-80cc-ad4f400259ba None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:24.747950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 443/873] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:24.749816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8b073-de87-4fc7-a9b5-8c725d68a3d8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:24.750310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8b073-de87-4fc7-a9b5-8c725d68a3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:24.750827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8b073-de87-4fc7-a9b5-8c725d68a3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:24.765644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:24.779532 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-78a8b073-de87-4fc7-a9b5-8c725d68a3d8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:24.781363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-78a8b073-de87-4fc7-a9b5-8c725d68a3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:24.868648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a54fc3-67fd-4c1f-82c4-f2a6b58ef005 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:24.893610 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 444/874] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:24.911912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4b1f9-5210-4630-a887-f9fa38e116c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:24.913106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4b1f9-5210-4630-a887-f9fa38e116c5 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:24.913448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4b1f9-5210-4630-a887-f9fa38e116c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:24.913730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4b1f9-5210-4630-a887-f9fa38e116c5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:24.927096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4b1f9-5210-4630-a887-f9fa38e116c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:24.931255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8b073-de87-4fc7-a9b5-8c725d68a3d8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-eeb4b1f9-5210-4630-a887-f9fa38e116c5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:24.936598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:24.938179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:13:24.940145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:24.942664 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 431/875] 10.222.0.93 () {56 vars in 1120 bytes} [Fri Jun 3 15:13:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:24.944264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4b1f9-5210-4630-a887-f9fa38e116c5 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:24.967315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:25.083305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8b073-de87-4fc7-a9b5-8c725d68a3d8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:25.084624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4b1f9-5210-4630-a887-f9fa38e116c5 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:25.088285 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:25.124728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ac5e2-0a73-4df6-950f-e5f00efff548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:25.125122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ac5e2-0a73-4df6-950f-e5f00efff548 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:25.125349 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ac5e2-0a73-4df6-950f-e5f00efff548 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:25.125627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ac5e2-0a73-4df6-950f-e5f00efff548 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:25.126376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ac5e2-0a73-4df6-950f-e5f00efff548 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:25.126731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ac5e2-0a73-4df6-950f-e5f00efff548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:25.128235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ac5e2-0a73-4df6-950f-e5f00efff548 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:25.130072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ba086-5d6e-445f-986d-fd2a5217c8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:25.131198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ba086-5d6e-445f-986d-fd2a5217c8c0 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:25.131695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ba086-5d6e-445f-986d-fd2a5217c8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:25.132167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ba086-5d6e-445f-986d-fd2a5217c8c0 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:25.133096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ba086-5d6e-445f-986d-fd2a5217c8c0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:25.135424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ba086-5d6e-445f-986d-fd2a5217c8c0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:25.138567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 432/876] 10.222.0.93 () {56 vars in 1115 bytes} [Fri Jun 3 15:13:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:25.144358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ac5e2-0a73-4df6-950f-e5f00efff548 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:25.173123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 445/877] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:13:24 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:13:25.303063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ac5e2-0a73-4df6-950f-e5f00efff548 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:25.349654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c86b39-b79b-47c3-8519-6b35b4bd2d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:25.351082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c86b39-b79b-47c3-8519-6b35b4bd2d14 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:25.351747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c86b39-b79b-47c3-8519-6b35b4bd2d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:25.352232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c86b39-b79b-47c3-8519-6b35b4bd2d14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:25.365257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c86b39-b79b-47c3-8519-6b35b4bd2d14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:25.367400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c86b39-b79b-47c3-8519-6b35b4bd2d14 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-38c86b39-b79b-47c3-8519-6b35b4bd2d14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:25.387364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:25.388317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:25.389203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:25.389203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:25.389203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:25.389203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:25.391532 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 433/878] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:25.396804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ba086-5d6e-445f-986d-fd2a5217c8c0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:25.482404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:25.508076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ba086-5d6e-445f-986d-fd2a5217c8c0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:25.518519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6686b3d-7f43-4f96-85b0-65c8c2d8a3ce None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:25.518837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6686b3d-7f43-4f96-85b0-65c8c2d8a3ce None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:25.519118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6686b3d-7f43-4f96-85b0-65c8c2d8a3ce None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:25.520634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 434/879] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:25.526261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2182e2f-e96e-4a5e-ac7e-5b64e82f41d0 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:25.526567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2182e2f-e96e-4a5e-ac7e-5b64e82f41d0 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:25.526839 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2182e2f-e96e-4a5e-ac7e-5b64e82f41d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:25.545217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d2182e2f-e96e-4a5e-ac7e-5b64e82f41d0 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:25.546412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d2182e2f-e96e-4a5e-ac7e-5b64e82f41d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:25.551462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23012b2d-a671-43f9-8986-fc0d93807ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:25.552748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012b2d-a671-43f9-8986-fc0d93807ebf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:25.553262 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012b2d-a671-43f9-8986-fc0d93807ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:25.553745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012b2d-a671-43f9-8986-fc0d93807ebf None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:25.554807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23012b2d-a671-43f9-8986-fc0d93807ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:25.555357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23012b2d-a671-43f9-8986-fc0d93807ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:25.557519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23012b2d-a671-43f9-8986-fc0d93807ebf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:25.560823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:25.573788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 446/880] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:25.579330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23012b2d-a671-43f9-8986-fc0d93807ebf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:25.652865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 435/881] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:25 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:25.683357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2182e2f-e96e-4a5e-ac7e-5b64e82f41d0 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:25.691548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23012b2d-a671-43f9-8986-fc0d93807ebf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:25.730982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c8b79-d626-44cc-92d4-d044f576aa92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:25.732101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c8b79-d626-44cc-92d4-d044f576aa92 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:25.732559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c8b79-d626-44cc-92d4-d044f576aa92 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:25.732989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c8b79-d626-44cc-92d4-d044f576aa92 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:25.741628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c8b79-d626-44cc-92d4-d044f576aa92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:25.743434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c8b79-d626-44cc-92d4-d044f576aa92 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:25.746169 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d63c8b79-d626-44cc-92d4-d044f576aa92 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:25.763225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:25.764280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:25.765259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:25.765259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:25.765259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:25.765259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:25.765259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:25.765259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:25.765259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:25.765259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:25.765259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:25.765259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:25.772396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 447/882] 10.222.0.93 () {56 vars in 1117 bytes} [Fri Jun 3 15:13:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:25.774204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c8b79-d626-44cc-92d4-d044f576aa92 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:25.825220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2182e2f-e96e-4a5e-ac7e-5b64e82f41d0 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:25.877387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cab17-898f-4812-92cb-47ace461455a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:25.878487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cab17-898f-4812-92cb-47ace461455a None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:25.879029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cab17-898f-4812-92cb-47ace461455a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:25.879510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5cab17-898f-4812-92cb-47ace461455a None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:25.881052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cab17-898f-4812-92cb-47ace461455a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:25.883028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5cab17-898f-4812-92cb-47ace461455a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:25.894359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c8b79-d626-44cc-92d4-d044f576aa92 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:25.903714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 436/883] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:13:25 2022] POST /identity/v3/services => generated 269 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:13:25.955196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446da07c-b9f4-486c-87b7-9317997c46cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:25.955915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446da07c-b9f4-486c-87b7-9317997c46cc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:25.956196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446da07c-b9f4-486c-87b7-9317997c46cc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:25.956470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446da07c-b9f4-486c-87b7-9317997c46cc None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:25.957324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446da07c-b9f4-486c-87b7-9317997c46cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:25.957736 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446da07c-b9f4-486c-87b7-9317997c46cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:25.959269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446da07c-b9f4-486c-87b7-9317997c46cc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:25.993888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 448/884] 10.222.0.93 () {56 vars in 1149 bytes} [Fri Jun 3 15:13:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:26.000007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cab17-898f-4812-92cb-47ace461455a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:26.020645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:26.122486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5cab17-898f-4812-92cb-47ace461455a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:26.161162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92042f94-e146-4037-b5ad-192a3551c232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:26.161778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042f94-e146-4037-b5ad-192a3551c232 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:26.162060 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042f94-e146-4037-b5ad-192a3551c232 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:26.162665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92042f94-e146-4037-b5ad-192a3551c232 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:26.238123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92042f94-e146-4037-b5ad-192a3551c232 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:26.239631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92042f94-e146-4037-b5ad-192a3551c232 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:26.242679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:26.351363 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 437/885] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:25 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:13:26.356850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446da07c-b9f4-486c-87b7-9317997c46cc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:26.365885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466489b-48cd-44d6-a600-d5da391b6b1b None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:26.367735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466489b-48cd-44d6-a600-d5da391b6b1b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:26.368546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466489b-48cd-44d6-a600-d5da391b6b1b None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:26.378660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 438/886] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:26.382578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b85ab5-cf81-4809-993f-201f6b9b1d56 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:26.382892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b85ab5-cf81-4809-993f-201f6b9b1d56 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:26.383143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b85ab5-cf81-4809-993f-201f6b9b1d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:26.412675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-55b85ab5-cf81-4809-993f-201f6b9b1d56 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:26.413988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-55b85ab5-cf81-4809-993f-201f6b9b1d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:26.439332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:26.502197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 439/887] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:26.514501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446da07c-b9f4-486c-87b7-9317997c46cc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:26.527637 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b85ab5-cf81-4809-993f-201f6b9b1d56 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:26.559910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba614a9-18cd-4309-8b7e-fe965f869322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:26.560780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba614a9-18cd-4309-8b7e-fe965f869322 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:26.561003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba614a9-18cd-4309-8b7e-fe965f869322 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:26.561729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba614a9-18cd-4309-8b7e-fe965f869322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:26.573246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba614a9-18cd-4309-8b7e-fe965f869322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:26.574884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba614a9-18cd-4309-8b7e-fe965f869322 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:26.580032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bba614a9-18cd-4309-8b7e-fe965f869322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:26.597932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:26.598966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:26.599877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:26.599877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:26.599877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:26.599877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:26.604262 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 449/888] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:26.605994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba614a9-18cd-4309-8b7e-fe965f869322 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:26.680783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b85ab5-cf81-4809-993f-201f6b9b1d56 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:26.718358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba614a9-18cd-4309-8b7e-fe965f869322 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:26.720964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:26.722486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8207efc-c028-4de1-b43b-002c0c0051cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:26.723762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207efc-c028-4de1-b43b-002c0c0051cc None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:26.724307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207efc-c028-4de1-b43b-002c0c0051cc None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:26.724808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8207efc-c028-4de1-b43b-002c0c0051cc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:26.725778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8207efc-c028-4de1-b43b-002c0c0051cc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:26.727620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8207efc-c028-4de1-b43b-002c0c0051cc None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-b8207efc-c028-4de1-b43b-002c0c0051cc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:26.741064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:26.742265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:26.743269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:26.751375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 440/889] 10.222.0.93 () {56 vars in 1126 bytes} [Fri Jun 3 15:13:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:26.753366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8207efc-c028-4de1-b43b-002c0c0051cc None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:26.757180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731da84-2bd2-4025-af6a-435843380e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:26.757974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731da84-2bd2-4025-af6a-435843380e8d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:26.758263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731da84-2bd2-4025-af6a-435843380e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:26.758557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731da84-2bd2-4025-af6a-435843380e8d None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:26.760526 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731da84-2bd2-4025-af6a-435843380e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:26.760526 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731da84-2bd2-4025-af6a-435843380e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:26.761692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731da84-2bd2-4025-af6a-435843380e8d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:26.780111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 450/890] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:26.781546 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731da84-2bd2-4025-af6a-435843380e8d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:26.868036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:26.873167 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8207efc-c028-4de1-b43b-002c0c0051cc None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:26.910656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731da84-2bd2-4025-af6a-435843380e8d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:26.913428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcb57f-9f4d-4d0c-b782-f20ef0c00234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:26.914451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcb57f-9f4d-4d0c-b782-f20ef0c00234 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:26.915055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcb57f-9f4d-4d0c-b782-f20ef0c00234 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:26.915560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcb57f-9f4d-4d0c-b782-f20ef0c00234 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:26.917110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcb57f-9f4d-4d0c-b782-f20ef0c00234 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:26.917646 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcb57f-9f4d-4d0c-b782-f20ef0c00234 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:26.919295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcb57f-9f4d-4d0c-b782-f20ef0c00234 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:26.930005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 441/891] 10.222.0.93 () {56 vars in 1122 bytes} [Fri Jun 3 15:13:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:26.932456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcb57f-9f4d-4d0c-b782-f20ef0c00234 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:26.951109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13530d5a-29cf-4a69-9fcf-b849452c5a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:26.952141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13530d5a-29cf-4a69-9fcf-b849452c5a1f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:26.952438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13530d5a-29cf-4a69-9fcf-b849452c5a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:26.952712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13530d5a-29cf-4a69-9fcf-b849452c5a1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:26.960868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13530d5a-29cf-4a69-9fcf-b849452c5a1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:26.962512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13530d5a-29cf-4a69-9fcf-b849452c5a1f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-13530d5a-29cf-4a69-9fcf-b849452c5a1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:26.979212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:26.980186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:26.981067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:26.981067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:26.981067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:26.981067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:26.981067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:26.981067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:26.981067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:26.981067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:26.985369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 451/892] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:26.987151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13530d5a-29cf-4a69-9fcf-b849452c5a1f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:27.017462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:27.051989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcb57f-9f4d-4d0c-b782-f20ef0c00234 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:27.091421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd62d5-1cbb-4bc0-b406-17cdfcacc261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:27.092466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd62d5-1cbb-4bc0-b406-17cdfcacc261 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:27.092949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd62d5-1cbb-4bc0-b406-17cdfcacc261 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:27.093449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd62d5-1cbb-4bc0-b406-17cdfcacc261 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:27.094456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd62d5-1cbb-4bc0-b406-17cdfcacc261 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:27.095016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd62d5-1cbb-4bc0-b406-17cdfcacc261 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:27.096586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd62d5-1cbb-4bc0-b406-17cdfcacc261 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:27.106566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13530d5a-29cf-4a69-9fcf-b849452c5a1f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:27.106947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 442/893] 10.222.0.93 () {56 vars in 1122 bytes} [Fri Jun 3 15:13:26 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:27.148791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2279d63-f3bd-421b-8aa3-01127c27e169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:27.149537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2279d63-f3bd-421b-8aa3-01127c27e169 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:27.149828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2279d63-f3bd-421b-8aa3-01127c27e169 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:27.150111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2279d63-f3bd-421b-8aa3-01127c27e169 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:27.151489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2279d63-f3bd-421b-8aa3-01127c27e169 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:27.151803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2279d63-f3bd-421b-8aa3-01127c27e169 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:27.153273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2279d63-f3bd-421b-8aa3-01127c27e169 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:27.173048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 452/894] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:27.174616 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd62d5-1cbb-4bc0-b406-17cdfcacc261 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:27.209860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:27.294428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd62d5-1cbb-4bc0-b406-17cdfcacc261 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:27.332355 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a1b75-a026-4c47-94cf-aa9f86f5a384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:27.333024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a1b75-a026-4c47-94cf-aa9f86f5a384 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:27.333309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a1b75-a026-4c47-94cf-aa9f86f5a384 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:27.333589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a1b75-a026-4c47-94cf-aa9f86f5a384 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:27.334439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a1b75-a026-4c47-94cf-aa9f86f5a384 None admin] RBAC: Adding query filter params (role.id=ca62ca555176433484fd406e5442b7d2, scope.project.id=23546323ca33412fb388afe308a97499, user.id=0603019b08d94980863f6612907601a9) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:27.334858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a1b75-a026-4c47-94cf-aa9f86f5a384 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:27.336335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a1b75-a026-4c47-94cf-aa9f86f5a384 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:27.350685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 443/895] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:27 2022] GET /identity/v3/role_assignments?user.id=0603019b08d94980863f6612907601a9&scope.project.id=23546323ca33412fb388afe308a97499&role.id=ca62ca555176433484fd406e5442b7d2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:28.191699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075f601-2446-4ccc-a30f-20667ef3e84a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:28.192247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075f601-2446-4ccc-a30f-20667ef3e84a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:28.192247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075f601-2446-4ccc-a30f-20667ef3e84a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:28.193942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 453/896] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:28.200376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e0414-7e00-484d-9854-31cdd5c5eabb None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:28.200998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e0414-7e00-484d-9854-31cdd5c5eabb None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:28.200998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e0414-7e00-484d-9854-31cdd5c5eabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:28.218750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-3b0e0414-7e00-484d-9854-31cdd5c5eabb None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:28.219948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3b0e0414-7e00-484d-9854-31cdd5c5eabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:28.288331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 444/897] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:28.311756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075f601-2446-4ccc-a30f-20667ef3e84a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:28.403791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:28.423336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075f601-2446-4ccc-a30f-20667ef3e84a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:28.457850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c1175-a280-41f3-8b77-8828756d1e10 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:28.458134 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c1175-a280-41f3-8b77-8828756d1e10 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:28.458344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c1175-a280-41f3-8b77-8828756d1e10 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:28.460043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 445/898] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:28.462444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb01c18-3d54-4bbb-842e-2533ed73514f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:28.463662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb01c18-3d54-4bbb-842e-2533ed73514f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:28.463953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb01c18-3d54-4bbb-842e-2533ed73514f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:28.464231 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb01c18-3d54-4bbb-842e-2533ed73514f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:28.465361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb01c18-3d54-4bbb-842e-2533ed73514f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:28.466388 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6f529-a470-4709-8cf4-d85651221fa3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:28.466738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6f529-a470-4709-8cf4-d85651221fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:28.466844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb01c18-3d54-4bbb-842e-2533ed73514f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:28.467021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6f529-a470-4709-8cf4-d85651221fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-0cb01c18-3d54-4bbb-842e-2533ed73514f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:28.480272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:28.481298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:28.482151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 454/899] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:13:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:28.487535 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb01c18-3d54-4bbb-842e-2533ed73514f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:28.491306 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-a2e6f529-a470-4709-8cf4-d85651221fa3 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:28.492501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a2e6f529-a470-4709-8cf4-d85651221fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:28.514414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:28.591045 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 446/900] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:28.615883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb01c18-3d54-4bbb-842e-2533ed73514f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:28.618728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6f529-a470-4709-8cf4-d85651221fa3 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:28.682420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f5657-e517-492a-b268-6f281a99e2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:28.684181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f5657-e517-492a-b268-6f281a99e2da None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:28.684760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f5657-e517-492a-b268-6f281a99e2da None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:28.685657 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f5657-e517-492a-b268-6f281a99e2da None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:28.687053 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f5657-e517-492a-b268-6f281a99e2da None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:28.687969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f5657-e517-492a-b268-6f281a99e2da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:28.691971 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f5657-e517-492a-b268-6f281a99e2da None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:28.714473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 455/901] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:13:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:28.716638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f5657-e517-492a-b268-6f281a99e2da None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:28.750916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:28.765449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6f529-a470-4709-8cf4-d85651221fa3 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:28.814351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66970967-c40d-4676-92ce-9756617f13f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:28.815561 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66970967-c40d-4676-92ce-9756617f13f7 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:28.816079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66970967-c40d-4676-92ce-9756617f13f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:28.816578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66970967-c40d-4676-92ce-9756617f13f7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:28.817439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66970967-c40d-4676-92ce-9756617f13f7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:28.819061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66970967-c40d-4676-92ce-9756617f13f7 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-66970967-c40d-4676-92ce-9756617f13f7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:28.832591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:28.833597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 15:13:28.834441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:28.842850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 447/902] 10.222.0.93 () {56 vars in 1132 bytes} [Fri Jun 3 15:13:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:28.844482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66970967-c40d-4676-92ce-9756617f13f7 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:28.863866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:28.926373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f5657-e517-492a-b268-6f281a99e2da None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:28.967528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e40e1-0395-4620-8405-2ec1df65dc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:28.968606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e40e1-0395-4620-8405-2ec1df65dc40 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:28.969099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e40e1-0395-4620-8405-2ec1df65dc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:28.969580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e40e1-0395-4620-8405-2ec1df65dc40 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:28.970572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e40e1-0395-4620-8405-2ec1df65dc40 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:28.971087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e40e1-0395-4620-8405-2ec1df65dc40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:28.972671 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e40e1-0395-4620-8405-2ec1df65dc40 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:28.980425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 456/903] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:13:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:28.988051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e40e1-0395-4620-8405-2ec1df65dc40 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:29.035765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66970967-c40d-4676-92ce-9756617f13f7 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:29.076735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:29.103820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e40e1-0395-4620-8405-2ec1df65dc40 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:29.110879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7330f2-8a57-443c-8471-a0d40f6c83ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:29.112682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7330f2-8a57-443c-8471-a0d40f6c83ca None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:29.113463 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7330f2-8a57-443c-8471-a0d40f6c83ca None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:29.114253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7330f2-8a57-443c-8471-a0d40f6c83ca None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:29.115983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7330f2-8a57-443c-8471-a0d40f6c83ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:29.117087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7330f2-8a57-443c-8471-a0d40f6c83ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:29.120271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7330f2-8a57-443c-8471-a0d40f6c83ca None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:29.134896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 448/904] 10.222.0.93 () {56 vars in 1126 bytes} [Fri Jun 3 15:13:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:29.140627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7330f2-8a57-443c-8471-a0d40f6c83ca None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:29.141643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b13e9f-ac56-4793-9cec-201ef784914f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:29.142313 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b13e9f-ac56-4793-9cec-201ef784914f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:29.142624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b13e9f-ac56-4793-9cec-201ef784914f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:29.142902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b13e9f-ac56-4793-9cec-201ef784914f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:29.143701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b13e9f-ac56-4793-9cec-201ef784914f None admin] RBAC: Adding query filter params (interface=public, service_id=024772b71cf44445a499928c5607c2e3, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:29.144011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b13e9f-ac56-4793-9cec-201ef784914f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:29.145394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b13e9f-ac56-4793-9cec-201ef784914f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:29.156210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 457/905] 10.222.0.93 () {56 vars in 1251 bytes} [Fri Jun 3 15:13:28 2022] GET /identity/v3/endpoints?service_id=024772b71cf44445a499928c5607c2e3&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:29.163292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b13e9f-ac56-4793-9cec-201ef784914f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:29.260438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:29.283480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b13e9f-ac56-4793-9cec-201ef784914f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:29.291754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7330f2-8a57-443c-8471-a0d40f6c83ca None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:29.322879 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96f477-b19b-45c3-b17d-b7f7d9081045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:29.323679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96f477-b19b-45c3-b17d-b7f7d9081045 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:29.323966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96f477-b19b-45c3-b17d-b7f7d9081045 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:29.324246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96f477-b19b-45c3-b17d-b7f7d9081045 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:29.324937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96f477-b19b-45c3-b17d-b7f7d9081045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:29.326375 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a96f477-b19b-45c3-b17d-b7f7d9081045 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:29.333444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d477eb-f45a-4dbd-9fa5-13ee0b1659b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:29.334710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d477eb-f45a-4dbd-9fa5-13ee0b1659b2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:29.335677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d477eb-f45a-4dbd-9fa5-13ee0b1659b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:29.336459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d477eb-f45a-4dbd-9fa5-13ee0b1659b2 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:29.338933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d477eb-f45a-4dbd-9fa5-13ee0b1659b2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:29.339858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d477eb-f45a-4dbd-9fa5-13ee0b1659b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:29.342219 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d477eb-f45a-4dbd-9fa5-13ee0b1659b2 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:29.342745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 458/906] 10.222.0.93 () {56 vars in 1087 bytes} [Fri Jun 3 15:13:29 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:13:29.349597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 449/907] 10.222.0.93 () {56 vars in 1126 bytes} [Fri Jun 3 15:13:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:29.356670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96f477-b19b-45c3-b17d-b7f7d9081045 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:29.361104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:29.487319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a96f477-b19b-45c3-b17d-b7f7d9081045 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:29.531574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44509204-616b-4395-b5dd-9f1c68cb38e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:29.532179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509204-616b-4395-b5dd-9f1c68cb38e2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:29.532466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509204-616b-4395-b5dd-9f1c68cb38e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:29.532746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44509204-616b-4395-b5dd-9f1c68cb38e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:29.533574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44509204-616b-4395-b5dd-9f1c68cb38e2 None admin] RBAC: Adding query filter params (interface=public, service_id=e5661a573640438286909348c0e11eb7, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:29.533874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44509204-616b-4395-b5dd-9f1c68cb38e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:29.535310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44509204-616b-4395-b5dd-9f1c68cb38e2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:29.544916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 459/908] 10.222.0.93 () {56 vars in 1251 bytes} [Fri Jun 3 15:13:29 2022] GET /identity/v3/endpoints?service_id=e5661a573640438286909348c0e11eb7&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:29.550560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d477eb-f45a-4dbd-9fa5-13ee0b1659b2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:29.623596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47a07c-fcda-477d-9725-9c09c2c7601f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:29.623924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47a07c-fcda-477d-9725-9c09c2c7601f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:29.624157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47a07c-fcda-477d-9725-9c09c2c7601f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:29.626668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 460/909] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:29.635028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912145f4-0c44-418a-a336-c5722c7a5254 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:29.635028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912145f4-0c44-418a-a336-c5722c7a5254 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:29.635028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912145f4-0c44-418a-a336-c5722c7a5254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:29.655445 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-912145f4-0c44-418a-a336-c5722c7a5254 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:29.657286 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-912145f4-0c44-418a-a336-c5722c7a5254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:29.692834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d477eb-f45a-4dbd-9fa5-13ee0b1659b2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:29.739784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:29.743008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0804180-e415-4e45-9dd1-3dad9bd21b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:29.744030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0804180-e415-4e45-9dd1-3dad9bd21b8a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:29.745061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0804180-e415-4e45-9dd1-3dad9bd21b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:29.745558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0804180-e415-4e45-9dd1-3dad9bd21b8a None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:29.746522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0804180-e415-4e45-9dd1-3dad9bd21b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:29.748164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0804180-e415-4e45-9dd1-3dad9bd21b8a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:29.754837 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 461/910] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:29.756024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 450/911] 10.222.0.93 () {56 vars in 1087 bytes} [Fri Jun 3 15:13:29 2022] GET /identity/v3/services => generated 1268 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:13:29.776208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912145f4-0c44-418a-a336-c5722c7a5254 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:29.845957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5109-d239-4c6a-be16-47d352f862ec None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:29.846253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5109-d239-4c6a-be16-47d352f862ec None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:29.846795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc5109-d239-4c6a-be16-47d352f862ec None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:29.848774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 451/912] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:13:29.856107 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692153e-9ff9-4ec0-97dc-50574b4d4c1e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:29.856425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692153e-9ff9-4ec0-97dc-50574b4d4c1e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:29.856668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692153e-9ff9-4ec0-97dc-50574b4d4c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:29.869769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:29.877794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-b692153e-9ff9-4ec0-97dc-50574b4d4c1e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:29.879494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b692153e-9ff9-4ec0-97dc-50574b4d4c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:29.906234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912145f4-0c44-418a-a336-c5722c7a5254 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:29.946452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219c687-cb15-400a-aba5-21b08c35b6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:29.947799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219c687-cb15-400a-aba5-21b08c35b6d9 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:29.948078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219c687-cb15-400a-aba5-21b08c35b6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:29.948350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219c687-cb15-400a-aba5-21b08c35b6d9 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:29.949519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219c687-cb15-400a-aba5-21b08c35b6d9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:29.950579 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5219c687-cb15-400a-aba5-21b08c35b6d9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:29.968102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 462/913] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:13:29 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:29.968799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 452/914] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:29.992004 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219c687-cb15-400a-aba5-21b08c35b6d9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:30.065852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:30.123908 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219c687-cb15-400a-aba5-21b08c35b6d9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:30.166560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b385a24-2a2a-41c2-9b6d-342adf989fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:30.167445 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b385a24-2a2a-41c2-9b6d-342adf989fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:30.167777 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b385a24-2a2a-41c2-9b6d-342adf989fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:30.168061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b385a24-2a2a-41c2-9b6d-342adf989fc5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:30.180339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b385a24-2a2a-41c2-9b6d-342adf989fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-4b385a24-2a2a-41c2-9b6d-342adf989fc5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:30.188954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:30.190069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:13:30.191035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:30.194666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 463/915] 10.222.0.93 () {56 vars in 1120 bytes} [Fri Jun 3 15:13:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:30.196780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692153e-9ff9-4ec0-97dc-50574b4d4c1e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:30.318567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692153e-9ff9-4ec0-97dc-50574b4d4c1e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:30.356508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d62e2b-e9fc-4ad1-97fc-4104319be54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:30.357207 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d62e2b-e9fc-4ad1-97fc-4104319be54d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:30.357471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d62e2b-e9fc-4ad1-97fc-4104319be54d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:30.357748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d62e2b-e9fc-4ad1-97fc-4104319be54d None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:30.358504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d62e2b-e9fc-4ad1-97fc-4104319be54d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:30.358860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d62e2b-e9fc-4ad1-97fc-4104319be54d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:30.360262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d62e2b-e9fc-4ad1-97fc-4104319be54d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:30.362701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:30.374742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 453/916] 10.222.0.93 () {56 vars in 1115 bytes} [Fri Jun 3 15:13:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:30.377049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b385a24-2a2a-41c2-9b6d-342adf989fc5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:30.483896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b385a24-2a2a-41c2-9b6d-342adf989fc5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:30.502089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:30.523530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b09dfc-2dfe-448b-9411-52439e32456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:30.524320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b09dfc-2dfe-448b-9411-52439e32456d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:30.524599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b09dfc-2dfe-448b-9411-52439e32456d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:30.524869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b09dfc-2dfe-448b-9411-52439e32456d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:30.537971 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b09dfc-2dfe-448b-9411-52439e32456d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:30.539724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b09dfc-2dfe-448b-9411-52439e32456d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-12b09dfc-2dfe-448b-9411-52439e32456d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:30.559039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:30.560772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:30.562448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:30.562448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:30.562448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:30.562448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:30.562448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 464/917] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:30.566147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d62e2b-e9fc-4ad1-97fc-4104319be54d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:30.713612 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d62e2b-e9fc-4ad1-97fc-4104319be54d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:30.749958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef36f0-cfa1-4848-a63e-436bd04149a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:30.750653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef36f0-cfa1-4848-a63e-436bd04149a5 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:30.750940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef36f0-cfa1-4848-a63e-436bd04149a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:30.751217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef36f0-cfa1-4848-a63e-436bd04149a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:30.751946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef36f0-cfa1-4848-a63e-436bd04149a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:30.752255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef36f0-cfa1-4848-a63e-436bd04149a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:30.753638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef36f0-cfa1-4848-a63e-436bd04149a5 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:30.766974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 454/918] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:30.772260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b09dfc-2dfe-448b-9411-52439e32456d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:30.900477 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b09dfc-2dfe-448b-9411-52439e32456d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:30.937358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e56b9-9e4b-4edd-a512-a0562d6d2f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:30.938094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e56b9-9e4b-4edd-a512-a0562d6d2f4c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:30.938372 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e56b9-9e4b-4edd-a512-a0562d6d2f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:30.938678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e56b9-9e4b-4edd-a512-a0562d6d2f4c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:30.948406 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e56b9-9e4b-4edd-a512-a0562d6d2f4c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:30.950012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e56b9-9e4b-4edd-a512-a0562d6d2f4c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d56e56b9-9e4b-4edd-a512-a0562d6d2f4c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:30.965403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:30.966432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:30.967425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:30.967425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:30.967425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:30.967425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:30.967425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:30.967425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:30.967425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:30.967425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:30.967425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:30.967425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:30.967425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 465/919] 10.222.0.93 () {56 vars in 1117 bytes} [Fri Jun 3 15:13:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:30.972661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef36f0-cfa1-4848-a63e-436bd04149a5 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:31.036667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:31.084904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef36f0-cfa1-4848-a63e-436bd04149a5 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:31.122112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd5b32-1ced-4d24-b6fa-94b5f5f33e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:31.122943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd5b32-1ced-4d24-b6fa-94b5f5f33e5a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:31.123230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd5b32-1ced-4d24-b6fa-94b5f5f33e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:31.123519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd5b32-1ced-4d24-b6fa-94b5f5f33e5a None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:31.124455 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd5b32-1ced-4d24-b6fa-94b5f5f33e5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:31.124768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd5b32-1ced-4d24-b6fa-94b5f5f33e5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:31.126277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd5b32-1ced-4d24-b6fa-94b5f5f33e5a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:31.150080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 455/920] 10.222.0.93 () {56 vars in 1149 bytes} [Fri Jun 3 15:13:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:31.151448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e56b9-9e4b-4edd-a512-a0562d6d2f4c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:31.186944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:31.272970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e56b9-9e4b-4edd-a512-a0562d6d2f4c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:31.309448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5be01b-fb84-4acc-99fa-54666bfbc918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:31.310316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5be01b-fb84-4acc-99fa-54666bfbc918 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:31.310631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5be01b-fb84-4acc-99fa-54666bfbc918 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:31.310961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5be01b-fb84-4acc-99fa-54666bfbc918 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:31.322573 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5be01b-fb84-4acc-99fa-54666bfbc918 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:31.324717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5be01b-fb84-4acc-99fa-54666bfbc918 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-2c5be01b-fb84-4acc-99fa-54666bfbc918 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:31.342970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:31.345058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:31.347324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:31.347324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:31.347324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:31.347324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:31.350554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 466/921] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:31.350780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd5b32-1ced-4d24-b6fa-94b5f5f33e5a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:31.464298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd5b32-1ced-4d24-b6fa-94b5f5f33e5a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:31.485497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:31.507522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9567f-749c-44b3-b996-235aac9b9e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:31.508604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9567f-749c-44b3-b996-235aac9b9e4d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:31.509245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9567f-749c-44b3-b996-235aac9b9e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:31.510510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9567f-749c-44b3-b996-235aac9b9e4d None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:31.511751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9567f-749c-44b3-b996-235aac9b9e4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:31.512313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9567f-749c-44b3-b996-235aac9b9e4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:31.514146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9567f-749c-44b3-b996-235aac9b9e4d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:31.532855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 456/922] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:31.539375 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5be01b-fb84-4acc-99fa-54666bfbc918 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:31.637866 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:31.662922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5be01b-fb84-4acc-99fa-54666bfbc918 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:31.714420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad477a-ed3b-4a32-8248-af5222aa7cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:31.715749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad477a-ed3b-4a32-8248-af5222aa7cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:31.716240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad477a-ed3b-4a32-8248-af5222aa7cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:31.716692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad477a-ed3b-4a32-8248-af5222aa7cb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:31.727834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad477a-ed3b-4a32-8248-af5222aa7cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:31.731120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad477a-ed3b-4a32-8248-af5222aa7cb7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-afad477a-ed3b-4a32-8248-af5222aa7cb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:31.760887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:31.761874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:31.762814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:31.762814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:31.762814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:31.762814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:31.762814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:31.762814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:31.762814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:31.762814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:31.770751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 467/923] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:31.773295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9567f-749c-44b3-b996-235aac9b9e4d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:31.929142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc7984-ae4b-4dc2-9fad-9ceb150fb6b9 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:31.929142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc7984-ae4b-4dc2-9fad-9ceb150fb6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:31.929474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc7984-ae4b-4dc2-9fad-9ceb150fb6b9 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:31.936950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 468/924] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:31.940910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1515f-ccef-4421-b753-5455f286255e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:31.941407 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1515f-ccef-4421-b753-5455f286255e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:31.941914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1515f-ccef-4421-b753-5455f286255e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:31.947825 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9567f-749c-44b3-b996-235aac9b9e4d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:31.971785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-95b1515f-ccef-4421-b753-5455f286255e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:31.973858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-95b1515f-ccef-4421-b753-5455f286255e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:31.986503 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e07e76-dff2-4a35-b5ad-4ba76194798c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:31.987569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e07e76-dff2-4a35-b5ad-4ba76194798c None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:31.988064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e07e76-dff2-4a35-b5ad-4ba76194798c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:31.988550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e07e76-dff2-4a35-b5ad-4ba76194798c None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:31.989547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e07e76-dff2-4a35-b5ad-4ba76194798c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:31.990058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e07e76-dff2-4a35-b5ad-4ba76194798c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:31.991844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e07e76-dff2-4a35-b5ad-4ba76194798c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:32.006179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 457/925] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:32.017890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e07e76-dff2-4a35-b5ad-4ba76194798c None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:32.089808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 469/926] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:32.103275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:32.110220 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1515f-ccef-4421-b753-5455f286255e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:32.196889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:32.208363 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e07e76-dff2-4a35-b5ad-4ba76194798c None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:32.241566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1515f-ccef-4421-b753-5455f286255e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:32.260386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d069a2-5536-433e-9bfe-e2e0c26b1e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:32.262807 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d069a2-5536-433e-9bfe-e2e0c26b1e10 None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:32.263294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d069a2-5536-433e-9bfe-e2e0c26b1e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:32.263670 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d069a2-5536-433e-9bfe-e2e0c26b1e10 None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/0603019b08d94980863f6612907601a9/roles/ca62ca555176433484fd406e5442b7d2` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:32.267530 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d069a2-5536-433e-9bfe-e2e0c26b1e10 None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=0603019b08d94980863f6612907601a9, role_id=ca62ca555176433484fd406e5442b7d2)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:32.270565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d069a2-5536-433e-9bfe-e2e0c26b1e10 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:32.293366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 458/927] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:13:32 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/0603019b08d94980863f6612907601a9/roles/ca62ca555176433484fd406e5442b7d2 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:13:32.293366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65062f28-fd25-4ccd-aaad-b44af898f673 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:32.293366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65062f28-fd25-4ccd-aaad-b44af898f673 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:32.293366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65062f28-fd25-4ccd-aaad-b44af898f673 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:32.298593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 459/928] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:32.298593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b7d89-8b29-4fd2-9449-f82f395e69a7 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:32.298593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b7d89-8b29-4fd2-9449-f82f395e69a7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:32.298593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b7d89-8b29-4fd2-9449-f82f395e69a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:32.316844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffcacf-7ebb-49db-ad90-db0830e92811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:32.319164 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffcacf-7ebb-49db-ad90-db0830e92811 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:32.320262 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffcacf-7ebb-49db-ad90-db0830e92811 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:32.320772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffcacf-7ebb-49db-ad90-db0830e92811 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:32.321700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffcacf-7ebb-49db-ad90-db0830e92811 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:32.324213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffcacf-7ebb-49db-ad90-db0830e92811 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:32.325426 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-217b7d89-8b29-4fd2-9449-f82f395e69a7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:32.327108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-217b7d89-8b29-4fd2-9449-f82f395e69a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-0dffcacf-7ebb-49db-ad90-db0830e92811 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:32.340859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:32.342075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 03 15:13:32.342973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:32.347143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 470/929] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:13:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:32.358571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffcacf-7ebb-49db-ad90-db0830e92811 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:32.444874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 460/930] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:32.474015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b7d89-8b29-4fd2-9449-f82f395e69a7 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:32.515239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:32.560993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffcacf-7ebb-49db-ad90-db0830e92811 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:32.619419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:32.630260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf50cc3-ef2d-4acb-ab5b-aa7a8ab08949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:32.631553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf50cc3-ef2d-4acb-ab5b-aa7a8ab08949 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:32.632019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf50cc3-ef2d-4acb-ab5b-aa7a8ab08949 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:32.632456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf50cc3-ef2d-4acb-ab5b-aa7a8ab08949 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:32.634940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf50cc3-ef2d-4acb-ab5b-aa7a8ab08949 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:32.635855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf50cc3-ef2d-4acb-ab5b-aa7a8ab08949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:32.639124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf50cc3-ef2d-4acb-ab5b-aa7a8ab08949 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:32.654666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 471/931] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:13:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:32.658041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf50cc3-ef2d-4acb-ab5b-aa7a8ab08949 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:32.681165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b7d89-8b29-4fd2-9449-f82f395e69a7 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:32.728893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb19036a-7394-4181-9d34-4b2d4dd54fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:32.729644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19036a-7394-4181-9d34-4b2d4dd54fdc None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:32.729917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19036a-7394-4181-9d34-4b2d4dd54fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:32.730196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19036a-7394-4181-9d34-4b2d4dd54fdc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:32.730908 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19036a-7394-4181-9d34-4b2d4dd54fdc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:32.732309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb19036a-7394-4181-9d34-4b2d4dd54fdc None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-bb19036a-7394-4181-9d34-4b2d4dd54fdc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:32.747499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:32.748741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 03 15:13:32.749720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:32.757324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 461/932] 10.222.0.93 () {56 vars in 1132 bytes} [Fri Jun 3 15:13:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:32.759550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa413d-f911-439a-9447-d4260aec09fe None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:32.760049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa413d-f911-439a-9447-d4260aec09fe None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:32.760537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa413d-f911-439a-9447-d4260aec09fe None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:32.763764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 462/933] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:32.765216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa413d-f911-439a-9447-d4260aec09fe None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:32.832963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf50cc3-ef2d-4acb-ab5b-aa7a8ab08949 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:32.884120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77313afe-1731-4571-94ed-f2114c4f7a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:32.884933 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77313afe-1731-4571-94ed-f2114c4f7a8c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:32.885220 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77313afe-1731-4571-94ed-f2114c4f7a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:32.885492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77313afe-1731-4571-94ed-f2114c4f7a8c None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:32.886203 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77313afe-1731-4571-94ed-f2114c4f7a8c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:32.886513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77313afe-1731-4571-94ed-f2114c4f7a8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:32.888085 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77313afe-1731-4571-94ed-f2114c4f7a8c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:32.904168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 472/934] 10.222.0.93 () {56 vars in 1114 bytes} [Fri Jun 3 15:13:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:32.904168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cb9d0-93f3-4376-903d-05908bd72afc None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:32.904168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cb9d0-93f3-4376-903d-05908bd72afc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:32.904168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308cb9d0-93f3-4376-903d-05908bd72afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:32.907694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:32.926233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-308cb9d0-93f3-4376-903d-05908bd72afc None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:32.926610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa413d-f911-439a-9447-d4260aec09fe None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:32.928125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-308cb9d0-93f3-4376-903d-05908bd72afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:32.983487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e63af-89b1-419d-b1b1-f606cec4a38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:32.984080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e63af-89b1-419d-b1b1-f606cec4a38f None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:32.984365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e63af-89b1-419d-b1b1-f606cec4a38f None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:32.984645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e63af-89b1-419d-b1b1-f606cec4a38f None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:32.985366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e63af-89b1-419d-b1b1-f606cec4a38f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:32.985676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e63af-89b1-419d-b1b1-f606cec4a38f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:32.987113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e63af-89b1-419d-b1b1-f606cec4a38f None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:32.998576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 463/935] 10.222.0.93 () {56 vars in 1126 bytes} [Fri Jun 3 15:13:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:32.998576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e63af-89b1-419d-b1b1-f606cec4a38f None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:33.007252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:33.021971 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 473/936] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:33.022958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cb9d0-93f3-4376-903d-05908bd72afc None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:33.141113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e63af-89b1-419d-b1b1-f606cec4a38f None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:33.180155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308cb9d0-93f3-4376-903d-05908bd72afc None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:33.185051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8053d-92f1-4f58-a844-a4bcf4c78929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:33.185733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8053d-92f1-4f58-a844-a4bcf4c78929 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:33.186016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8053d-92f1-4f58-a844-a4bcf4c78929 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:33.186292 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8053d-92f1-4f58-a844-a4bcf4c78929 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:33.187184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8053d-92f1-4f58-a844-a4bcf4c78929 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:33.188692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8053d-92f1-4f58-a844-a4bcf4c78929 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:33.230739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 464/937] 10.222.0.93 () {60 vars in 1144 bytes} [Fri Jun 3 15:13:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:33.232394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8053d-92f1-4f58-a844-a4bcf4c78929 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:33.256037 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a863e93-af08-4380-9c88-1381a9c481f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:33.256837 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a863e93-af08-4380-9c88-1381a9c481f2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:33.257244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a863e93-af08-4380-9c88-1381a9c481f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:33.257598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a863e93-af08-4380-9c88-1381a9c481f2 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:33.258957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a863e93-af08-4380-9c88-1381a9c481f2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:33.259427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a863e93-af08-4380-9c88-1381a9c481f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:33.261304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a863e93-af08-4380-9c88-1381a9c481f2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:33.264947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:33.274568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 474/938] 10.222.0.93 () {56 vars in 1126 bytes} [Fri Jun 3 15:13:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:33.275811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a863e93-af08-4380-9c88-1381a9c481f2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:33.340097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:33.416043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a863e93-af08-4380-9c88-1381a9c481f2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:33.463632 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd90950-4ba5-449f-a6f9-7b6fa841b3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:33.463946 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd90950-4ba5-449f-a6f9-7b6fa841b3e7 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:33.464212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd90950-4ba5-449f-a6f9-7b6fa841b3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:33.465442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd90950-4ba5-449f-a6f9-7b6fa841b3e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:33.466258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd90950-4ba5-449f-a6f9-7b6fa841b3e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:33.467755 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd90950-4ba5-449f-a6f9-7b6fa841b3e7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:33.471115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8053d-92f1-4f58-a844-a4bcf4c78929 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:33.495893 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 475/939] 10.222.0.93 () {60 vars in 1144 bytes} [Fri Jun 3 15:13:33 2022] POST /identity/v3/endpoints => generated 347 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:33.558574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080f8c8-8539-41de-a381-8d49d4fc79d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:33.559970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080f8c8-8539-41de-a381-8d49d4fc79d5 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:33.560480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080f8c8-8539-41de-a381-8d49d4fc79d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:33.560973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080f8c8-8539-41de-a381-8d49d4fc79d5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:33.561908 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080f8c8-8539-41de-a381-8d49d4fc79d5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:33.563701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080f8c8-8539-41de-a381-8d49d4fc79d5 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-5080f8c8-8539-41de-a381-8d49d4fc79d5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:33.593844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:33.595890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:33.597549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:33.603006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 465/940] 10.222.0.93 () {56 vars in 1126 bytes} [Fri Jun 3 15:13:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:33.610134 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd90950-4ba5-449f-a6f9-7b6fa841b3e7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:33.781813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd90950-4ba5-449f-a6f9-7b6fa841b3e7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:33.793097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:33.840854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f7b30-1bb8-4ded-a577-ebe5ee72719f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:33.841645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f7b30-1bb8-4ded-a577-ebe5ee72719f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:33.842014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f7b30-1bb8-4ded-a577-ebe5ee72719f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:33.842398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f7b30-1bb8-4ded-a577-ebe5ee72719f None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:33.843765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f7b30-1bb8-4ded-a577-ebe5ee72719f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:33.844104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f7b30-1bb8-4ded-a577-ebe5ee72719f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:33.845902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f7b30-1bb8-4ded-a577-ebe5ee72719f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:33.853751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 476/941] 10.222.0.93 () {56 vars in 1122 bytes} [Fri Jun 3 15:13:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:33.860655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080f8c8-8539-41de-a381-8d49d4fc79d5 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:33.967670 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:34.017934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080f8c8-8539-41de-a381-8d49d4fc79d5 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:34.059711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee36b3-3eec-4bc7-a372-b42c1b8ca2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:34.060401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee36b3-3eec-4bc7-a372-b42c1b8ca2ce None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:34.060722 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee36b3-3eec-4bc7-a372-b42c1b8ca2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:34.060999 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee36b3-3eec-4bc7-a372-b42c1b8ca2ce None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:34.061733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee36b3-3eec-4bc7-a372-b42c1b8ca2ce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:34.062082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee36b3-3eec-4bc7-a372-b42c1b8ca2ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:34.063528 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee36b3-3eec-4bc7-a372-b42c1b8ca2ce None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:34.069858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 466/942] 10.222.0.93 () {56 vars in 1122 bytes} [Fri Jun 3 15:13:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:34.076502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f7b30-1bb8-4ded-a577-ebe5ee72719f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:34.209478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f7b30-1bb8-4ded-a577-ebe5ee72719f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:34.267917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b156788-1885-488f-9b22-5317b6a61f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:34.269025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b156788-1885-488f-9b22-5317b6a61f18 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:34.269541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b156788-1885-488f-9b22-5317b6a61f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:34.270024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b156788-1885-488f-9b22-5317b6a61f18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:34.271540 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b156788-1885-488f-9b22-5317b6a61f18 None admin] RBAC: Adding query filter params (interface=public, service_id=d70029901b1d45d68e701868a5ac4cb3, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:34.272086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b156788-1885-488f-9b22-5317b6a61f18 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:34.274663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b156788-1885-488f-9b22-5317b6a61f18 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:34.285066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 477/943] 10.222.0.93 () {56 vars in 1251 bytes} [Fri Jun 3 15:13:34 2022] GET /identity/v3/endpoints?service_id=d70029901b1d45d68e701868a5ac4cb3&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:34.290405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee36b3-3eec-4bc7-a372-b42c1b8ca2ce None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:34.455544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee36b3-3eec-4bc7-a372-b42c1b8ca2ce None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:34.518267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c07483-1005-4f8b-9a0f-11ac3e5de2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:34.519420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c07483-1005-4f8b-9a0f-11ac3e5de2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:34.519903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c07483-1005-4f8b-9a0f-11ac3e5de2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:34.520378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c07483-1005-4f8b-9a0f-11ac3e5de2e3 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:34.521615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c07483-1005-4f8b-9a0f-11ac3e5de2e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:34.524324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c07483-1005-4f8b-9a0f-11ac3e5de2e3 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:34.527212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:34.543697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 467/944] 10.222.0.93 () {56 vars in 1087 bytes} [Fri Jun 3 15:13:34 2022] GET /identity/v3/services => generated 1497 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:13:34.929456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bde6c-e114-4810-815c-b0f65011652a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:34.930226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bde6c-e114-4810-815c-b0f65011652a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:34.930226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bde6c-e114-4810-815c-b0f65011652a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:34.931595 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 478/945] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:34.937057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb843d1-9794-4690-8ed5-ecba4ef9b090 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:34.937361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb843d1-9794-4690-8ed5-ecba4ef9b090 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:34.937625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb843d1-9794-4690-8ed5-ecba4ef9b090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:34.956947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2cb843d1-9794-4690-8ed5-ecba4ef9b090 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:34.958165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2cb843d1-9794-4690-8ed5-ecba4ef9b090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:35.081542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 468/946] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:35.106001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bde6c-e114-4810-815c-b0f65011652a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:35.146354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:35.221097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bde6c-e114-4810-815c-b0f65011652a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:35.257924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88caa0-c48e-4bbf-aa4d-e6471c907d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:35.258778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88caa0-c48e-4bbf-aa4d-e6471c907d93 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:35.259064 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88caa0-c48e-4bbf-aa4d-e6471c907d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:35.259445 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88caa0-c48e-4bbf-aa4d-e6471c907d93 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:35.274810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88caa0-c48e-4bbf-aa4d-e6471c907d93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ce88caa0-c48e-4bbf-aa4d-e6471c907d93 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:35.283863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:35.285600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:35.288706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 479/947] 10.222.0.93 () {56 vars in 1120 bytes} [Fri Jun 3 15:13:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:35.294809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb843d1-9794-4690-8ed5-ecba4ef9b090 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:35.403332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb843d1-9794-4690-8ed5-ecba4ef9b090 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:35.433266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:35.442302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cf66a-b2a0-4f47-8478-b4e8c10f12c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:35.443228 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf66a-b2a0-4f47-8478-b4e8c10f12c2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:35.443661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf66a-b2a0-4f47-8478-b4e8c10f12c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:35.444087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf66a-b2a0-4f47-8478-b4e8c10f12c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:35.445080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cf66a-b2a0-4f47-8478-b4e8c10f12c2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:35.445536 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cf66a-b2a0-4f47-8478-b4e8c10f12c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:35.447662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cf66a-b2a0-4f47-8478-b4e8c10f12c2 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:35.463462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88caa0-c48e-4bbf-aa4d-e6471c907d93 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:35.464119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 469/948] 10.222.0.93 () {56 vars in 1115 bytes} [Fri Jun 3 15:13:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:35.573007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88caa0-c48e-4bbf-aa4d-e6471c907d93 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:35.580959 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:35.610102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8450018-90ba-4a1d-99a7-469d231c5cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:35.611013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8450018-90ba-4a1d-99a7-469d231c5cfc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:35.611297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8450018-90ba-4a1d-99a7-469d231c5cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:35.611576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8450018-90ba-4a1d-99a7-469d231c5cfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:35.622742 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8450018-90ba-4a1d-99a7-469d231c5cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:35.624234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8450018-90ba-4a1d-99a7-469d231c5cfc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a8450018-90ba-4a1d-99a7-469d231c5cfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:35.642880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:35.644394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:35.645862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:35.645862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:35.645862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:35.645862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:35.645862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 480/949] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:35.660635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cf66a-b2a0-4f47-8478-b4e8c10f12c2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:35.774332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cf66a-b2a0-4f47-8478-b4e8c10f12c2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:35.810749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f7172-48a3-4d5a-9292-6a3dc2939af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:35.811479 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f7172-48a3-4d5a-9292-6a3dc2939af8 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:35.811767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f7172-48a3-4d5a-9292-6a3dc2939af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:35.812092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f7172-48a3-4d5a-9292-6a3dc2939af8 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:35.812831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f7172-48a3-4d5a-9292-6a3dc2939af8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:35.813138 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f7172-48a3-4d5a-9292-6a3dc2939af8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:35.814548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f7172-48a3-4d5a-9292-6a3dc2939af8 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:35.827727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 470/950] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:35.833287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8450018-90ba-4a1d-99a7-469d231c5cfc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:35.973049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8450018-90ba-4a1d-99a7-469d231c5cfc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:36.021634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872012a-e32e-47bf-8fcc-64aeea7b01cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:36.022664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872012a-e32e-47bf-8fcc-64aeea7b01cb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:36.023270 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872012a-e32e-47bf-8fcc-64aeea7b01cb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:36.023642 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872012a-e32e-47bf-8fcc-64aeea7b01cb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:36.035651 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872012a-e32e-47bf-8fcc-64aeea7b01cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:36.037854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872012a-e32e-47bf-8fcc-64aeea7b01cb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f872012a-e32e-47bf-8fcc-64aeea7b01cb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:36.060670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:36.064273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:36.072796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:36.072796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:36.072796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:36.072796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:36.072796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:36.072796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:36.072796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:36.072796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:36.072796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 03 15:13:36.072796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:36.072796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 481/951] 10.222.0.93 () {56 vars in 1117 bytes} [Fri Jun 3 15:13:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:36.074303 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f7172-48a3-4d5a-9292-6a3dc2939af8 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:36.146369 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:36.229711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f7172-48a3-4d5a-9292-6a3dc2939af8 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:36.285509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a89d7-eb6c-4b7b-833d-57325e00a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:36.286659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a89d7-eb6c-4b7b-833d-57325e00a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:36.287415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a89d7-eb6c-4b7b-833d-57325e00a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:36.287916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a89d7-eb6c-4b7b-833d-57325e00a5c4 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:36.289546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a89d7-eb6c-4b7b-833d-57325e00a5c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:36.290072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a89d7-eb6c-4b7b-833d-57325e00a5c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:36.292668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a89d7-eb6c-4b7b-833d-57325e00a5c4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:36.327215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 471/952] 10.222.0.93 () {56 vars in 1149 bytes} [Fri Jun 3 15:13:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:36.332608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872012a-e32e-47bf-8fcc-64aeea7b01cb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:36.343077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:36.494509 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872012a-e32e-47bf-8fcc-64aeea7b01cb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:36.540760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce1094-f241-4915-8f5f-56d213dba07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:36.541907 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce1094-f241-4915-8f5f-56d213dba07d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:36.542656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce1094-f241-4915-8f5f-56d213dba07d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:36.543339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce1094-f241-4915-8f5f-56d213dba07d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:36.627614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce1094-f241-4915-8f5f-56d213dba07d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:36.629124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce1094-f241-4915-8f5f-56d213dba07d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:36.638948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:36.745073 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 482/953] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:36 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:13:36.753731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a89d7-eb6c-4b7b-833d-57325e00a5c4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:36.891630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d2804-357f-4a17-9234-458cd59fa21e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:36.892056 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d2804-357f-4a17-9234-458cd59fa21e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:36.892335 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d2804-357f-4a17-9234-458cd59fa21e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:36.893828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 483/954] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:13:36.899598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20893e-477f-4b3d-8aaa-0dd2eaf0d3a2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:36.900004 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20893e-477f-4b3d-8aaa-0dd2eaf0d3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:36.900245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20893e-477f-4b3d-8aaa-0dd2eaf0d3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:36.919503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1d20893e-477f-4b3d-8aaa-0dd2eaf0d3a2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:36.920755 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1d20893e-477f-4b3d-8aaa-0dd2eaf0d3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:36.929328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a89d7-eb6c-4b7b-833d-57325e00a5c4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:36.944669 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:36.978361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1485d7-af6c-4b1c-9429-10d384863ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:36.979344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1485d7-af6c-4b1c-9429-10d384863ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:36.979629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1485d7-af6c-4b1c-9429-10d384863ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:36.979905 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1485d7-af6c-4b1c-9429-10d384863ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:36.992024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1485d7-af6c-4b1c-9429-10d384863ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:36.993696 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1485d7-af6c-4b1c-9429-10d384863ee4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:37.009927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 484/955] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:37.013446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e4286-2964-4b12-ab30-d3f81b365e49 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:37.014310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e4286-2964-4b12-ab30-d3f81b365e49 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-eb1485d7-af6c-4b1c-9429-10d384863ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:37.014843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:37.016577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:37.019006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:37.019006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:37.019006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:37.019006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:37.020147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e4286-2964-4b12-ab30-d3f81b365e49 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:37.022101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 472/956] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:37.023018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1485d7-af6c-4b1c-9429-10d384863ee4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:37.030590 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 485/957] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:37 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:37.033695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec4935-fcb8-4cb4-99d9-7967ce486450 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:37.034199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec4935-fcb8-4cb4-99d9-7967ce486450 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:37.034732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec4935-fcb8-4cb4-99d9-7967ce486450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:37.063810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-18ec4935-fcb8-4cb4-99d9-7967ce486450 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:37.065809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-18ec4935-fcb8-4cb4-99d9-7967ce486450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:37.116668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:37.169307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1485d7-af6c-4b1c-9429-10d384863ee4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:37.191952 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 486/958] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:37.192813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec4935-fcb8-4cb4-99d9-7967ce486450 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:37.242834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0c0f0-e16d-4b9f-850e-370f8125b415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:37.244171 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0c0f0-e16d-4b9f-850e-370f8125b415 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:37.244704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0c0f0-e16d-4b9f-850e-370f8125b415 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:37.245504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0c0f0-e16d-4b9f-850e-370f8125b415 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:37.247231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0c0f0-e16d-4b9f-850e-370f8125b415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:37.247896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0c0f0-e16d-4b9f-850e-370f8125b415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:37.250146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0c0f0-e16d-4b9f-850e-370f8125b415 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:37.277475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 473/959] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:37.277475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0c0f0-e16d-4b9f-850e-370f8125b415 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:37.303423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:37.376394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec4935-fcb8-4cb4-99d9-7967ce486450 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:37.408077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0c0f0-e16d-4b9f-850e-370f8125b415 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:37.430458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b36650-d6ff-4d54-8813-f9e159af9f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:37.431859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b36650-d6ff-4d54-8813-f9e159af9f1c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:37.432317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b36650-d6ff-4d54-8813-f9e159af9f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:37.432662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b36650-d6ff-4d54-8813-f9e159af9f1c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:37.433327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b36650-d6ff-4d54-8813-f9e159af9f1c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:37.435727 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b36650-d6ff-4d54-8813-f9e159af9f1c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:37.446465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491dd174-87bc-4e0e-8e55-a7f4d1ee2108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:37.448168 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dd174-87bc-4e0e-8e55-a7f4d1ee2108 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:37.448652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dd174-87bc-4e0e-8e55-a7f4d1ee2108 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:37.449136 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491dd174-87bc-4e0e-8e55-a7f4d1ee2108 None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d9b36650-d6ff-4d54-8813-f9e159af9f1c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:37.449287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:37.450690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 15:13:37.451829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:37.453454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491dd174-87bc-4e0e-8e55-a7f4d1ee2108 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:37.453588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 487/960] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:37.453588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b36650-d6ff-4d54-8813-f9e159af9f1c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:37.453824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491dd174-87bc-4e0e-8e55-a7f4d1ee2108 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:37.455664 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: WARNING keystone.server.flask.common [None req-491dd174-87bc-4e0e-8e55-a7f4d1ee2108 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 03 15:13:37.478143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.sql.core [None req-491dd174-87bc-4e0e-8e55-a7f4d1ee2108 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 03 15:13:37.478143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 03 15:13:37.478143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [parameters: {'id': 'fdc5343b894a49c1a7b16a9772d2de8e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 03 15:13:37.478143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80221) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-491dd174-87bc-4e0e-8e55-a7f4d1ee2108 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fdc5343b894a49c1a7b16a9772d2de8e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fdc5343b894a49c1a7b16a9772d2de8e. Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application conn.query(q) Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 15:13:37.485841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result.read() Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 03 15:13:37.487943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application next(self.gen) Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application yield resource Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application next(self.gen) Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.session.rollback() Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.force_reraise() Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise self.value Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application session.commit() Jun 03 15:13:37.490129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._prepare_impl() Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.session.flush() Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._flush(objects) Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application compat.raise_( Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application flush_context.execute() Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 03 15:13:37.492089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rec.execute(self) Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 03 15:13:37.495318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application conn.query(q) Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result.read() Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 15:13:37.497190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [parameters: {'id': 'fdc5343b894a49c1a7b16a9772d2de8e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:37.499302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:37.501172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id fdc5343b894a49c1a7b16a9772d2de8e. Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 474/961] 10.222.0.93 () {60 vars in 1140 bytes} [Fri Jun 3 15:13:37 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd91bb-553d-4230-8ee4-a67e5f350259 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:37.503372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd91bb-553d-4230-8ee4-a67e5f350259 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:37.504912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd91bb-553d-4230-8ee4-a67e5f350259 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:37.505072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:37.511789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 475/962] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:37 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:37.512288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd91bb-553d-4230-8ee4-a67e5f350259 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:37.589770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b36650-d6ff-4d54-8813-f9e159af9f1c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:37.632865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2c3db-43d8-4334-8090-a087131821b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:37.633789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2c3db-43d8-4334-8090-a087131821b0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:37.633970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2c3db-43d8-4334-8090-a087131821b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:37.634834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2c3db-43d8-4334-8090-a087131821b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:37.640442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:37.643973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2c3db-43d8-4334-8090-a087131821b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:37.646235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed2c3db-43d8-4334-8090-a087131821b0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8ed2c3db-43d8-4334-8090-a087131821b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:37.666550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:37.668431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:37.670354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:37.670354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:37.670354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:37.670354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:37.670354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:37.670354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:37.670354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:37.670354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:37.670354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 488/963] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:37.670354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2c3db-43d8-4334-8090-a087131821b0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:37.680088 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd91bb-553d-4230-8ee4-a67e5f350259 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:37.723765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d781e9d-8db6-4faf-bd4d-24cd1916f560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:37.724449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781e9d-8db6-4faf-bd4d-24cd1916f560 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:37.724728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781e9d-8db6-4faf-bd4d-24cd1916f560 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:37.724998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781e9d-8db6-4faf-bd4d-24cd1916f560 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:37.725727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d781e9d-8db6-4faf-bd4d-24cd1916f560 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:37.726031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d781e9d-8db6-4faf-bd4d-24cd1916f560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:37.727629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d781e9d-8db6-4faf-bd4d-24cd1916f560 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:37.734524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 476/964] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:37.736242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9aeb2-166d-4bd3-815d-e6ddd3d36af6 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:37.736587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9aeb2-166d-4bd3-815d-e6ddd3d36af6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:37.736985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9aeb2-166d-4bd3-815d-e6ddd3d36af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:37.757342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c9b9aeb2-166d-4bd3-815d-e6ddd3d36af6 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:37.759044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c9b9aeb2-166d-4bd3-815d-e6ddd3d36af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:37.782437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:37.796422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed2c3db-43d8-4334-8090-a087131821b0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:37.842903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 477/965] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:37.844205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9aeb2-166d-4bd3-815d-e6ddd3d36af6 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:37.858511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1668f9-d3f3-46fd-bc22-c261664afed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:37.859852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1668f9-d3f3-46fd-bc22-c261664afed9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:37.860339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1668f9-d3f3-46fd-bc22-c261664afed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:37.860825 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1668f9-d3f3-46fd-bc22-c261664afed9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:37.873886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1668f9-d3f3-46fd-bc22-c261664afed9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:37.876680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1668f9-d3f3-46fd-bc22-c261664afed9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8b1668f9-d3f3-46fd-bc22-c261664afed9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:37.892330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:37.893301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:37.894496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:37.894496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:37.894496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:37.894496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:37.894496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:37.894496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:37.894496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:37.894496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:37.894496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 489/966] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:37.894496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1668f9-d3f3-46fd-bc22-c261664afed9 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:37.945702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:37.991487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9aeb2-166d-4bd3-815d-e6ddd3d36af6 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:38.028567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1668f9-d3f3-46fd-bc22-c261664afed9 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:38.033594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad7d2d-e9a4-4b19-ab50-290c7876a4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:38.034325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad7d2d-e9a4-4b19-ab50-290c7876a4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:38.034666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad7d2d-e9a4-4b19-ab50-290c7876a4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:38.034952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad7d2d-e9a4-4b19-ab50-290c7876a4f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:38.035762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad7d2d-e9a4-4b19-ab50-290c7876a4f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:38.036074 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad7d2d-e9a4-4b19-ab50-290c7876a4f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:38.037843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad7d2d-e9a4-4b19-ab50-290c7876a4f4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:38.051694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 478/967] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:38.052715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad7d2d-e9a4-4b19-ab50-290c7876a4f4 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:38.070254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393993e8-400c-4a7f-86bf-7f2f53fcaefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:38.070957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393993e8-400c-4a7f-86bf-7f2f53fcaefd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:38.071243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393993e8-400c-4a7f-86bf-7f2f53fcaefd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:38.071524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393993e8-400c-4a7f-86bf-7f2f53fcaefd None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:38.072259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393993e8-400c-4a7f-86bf-7f2f53fcaefd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:38.072568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393993e8-400c-4a7f-86bf-7f2f53fcaefd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:38.073961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393993e8-400c-4a7f-86bf-7f2f53fcaefd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:38.086785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 490/968] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:37 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:38.086785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393993e8-400c-4a7f-86bf-7f2f53fcaefd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:38.115678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:38.232819 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad7d2d-e9a4-4b19-ab50-290c7876a4f4 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:38.270424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393993e8-400c-4a7f-86bf-7f2f53fcaefd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:38.272459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:38.284701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5b961-2f11-45ca-8837-96a807ea13dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:38.286192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5b961-2f11-45ca-8837-96a807ea13dd None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:38.286788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5b961-2f11-45ca-8837-96a807ea13dd None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:38.287605 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5b961-2f11-45ca-8837-96a807ea13dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:38.288717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5b961-2f11-45ca-8837-96a807ea13dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:38.291931 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5b961-2f11-45ca-8837-96a807ea13dd None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-1ff5b961-2f11-45ca-8837-96a807ea13dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:38.306901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:38.308180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:38.309185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 479/969] 10.222.0.93 () {56 vars in 1126 bytes} [Fri Jun 3 15:13:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:38.314029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5b961-2f11-45ca-8837-96a807ea13dd None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:38.319779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930ca38-7bd2-47a5-9f9a-3343e1174c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:38.320451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930ca38-7bd2-47a5-9f9a-3343e1174c43 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:38.320732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930ca38-7bd2-47a5-9f9a-3343e1174c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:38.321011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930ca38-7bd2-47a5-9f9a-3343e1174c43 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:38.321772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930ca38-7bd2-47a5-9f9a-3343e1174c43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:38.322077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930ca38-7bd2-47a5-9f9a-3343e1174c43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:38.323907 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930ca38-7bd2-47a5-9f9a-3343e1174c43 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:38.343367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 491/970] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:38 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:38.344308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930ca38-7bd2-47a5-9f9a-3343e1174c43 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:38.450780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:38.471538 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5b961-2f11-45ca-8837-96a807ea13dd None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:38.481092 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930ca38-7bd2-47a5-9f9a-3343e1174c43 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:38.520663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cf285-8357-485d-83ee-a8873813063e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:38.525343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cf285-8357-485d-83ee-a8873813063e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:38.525343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cf285-8357-485d-83ee-a8873813063e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:38.525343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cf285-8357-485d-83ee-a8873813063e None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:38.525343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cf285-8357-485d-83ee-a8873813063e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:38.525343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cf285-8357-485d-83ee-a8873813063e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:38.525343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cf285-8357-485d-83ee-a8873813063e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:38.530978 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe84282-bffd-45d8-bee4-60d4818fc0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:38.531667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe84282-bffd-45d8-bee4-60d4818fc0ae None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:38.531945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe84282-bffd-45d8-bee4-60d4818fc0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:38.532218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe84282-bffd-45d8-bee4-60d4818fc0ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:38.533077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe84282-bffd-45d8-bee4-60d4818fc0ae None admin] RBAC: Adding query filter params (role.id=ca62ca555176433484fd406e5442b7d2, scope.project.id=23546323ca33412fb388afe308a97499, user.id=0603019b08d94980863f6612907601a9) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:38.533380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe84282-bffd-45d8-bee4-60d4818fc0ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:38.534944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe84282-bffd-45d8-bee4-60d4818fc0ae None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:38.541853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 492/971] 10.222.0.93 () {56 vars in 1122 bytes} [Fri Jun 3 15:13:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:38.542480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 480/972] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:38 2022] GET /identity/v3/role_assignments?user.id=0603019b08d94980863f6612907601a9&scope.project.id=23546323ca33412fb388afe308a97499&role.id=ca62ca555176433484fd406e5442b7d2 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:38.553627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cf285-8357-485d-83ee-a8873813063e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:38.720606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cf285-8357-485d-83ee-a8873813063e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:38.793981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad01186-ea10-4b3a-a32e-d46c14084d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:38.795369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad01186-ea10-4b3a-a32e-d46c14084d8c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:38.796003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad01186-ea10-4b3a-a32e-d46c14084d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:38.796634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad01186-ea10-4b3a-a32e-d46c14084d8c None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:38.798093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad01186-ea10-4b3a-a32e-d46c14084d8c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:38.798819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad01186-ea10-4b3a-a32e-d46c14084d8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:38.801543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad01186-ea10-4b3a-a32e-d46c14084d8c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:38.809411 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 493/973] 10.222.0.93 () {56 vars in 1122 bytes} [Fri Jun 3 15:13:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:38.821075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe84282-bffd-45d8-bee4-60d4818fc0ae None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:38.903053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:38.965379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe84282-bffd-45d8-bee4-60d4818fc0ae None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:39.007621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910a076-6206-4a75-b4d0-3104dc0c7e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:39.008297 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910a076-6206-4a75-b4d0-3104dc0c7e41 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:39.008585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910a076-6206-4a75-b4d0-3104dc0c7e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:39.008871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910a076-6206-4a75-b4d0-3104dc0c7e41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:39.010201 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910a076-6206-4a75-b4d0-3104dc0c7e41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:39.011626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910a076-6206-4a75-b4d0-3104dc0c7e41 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:39.043360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 481/974] 10.222.0.93 () {60 vars in 1144 bytes} [Fri Jun 3 15:13:38 2022] POST /identity/v3/endpoints => generated 349 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:40.686855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fa96b-26bc-4315-8bf7-2cd8f40c97d2 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:40.687357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fa96b-26bc-4315-8bf7-2cd8f40c97d2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:40.687469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69fa96b-26bc-4315-8bf7-2cd8f40c97d2 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:40.689076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 494/975] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:40.695970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e57a72-4de8-402c-966c-217a767f9bf3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:40.696399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e57a72-4de8-402c-966c-217a767f9bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:40.697269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e57a72-4de8-402c-966c-217a767f9bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:40.715083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-01e57a72-4de8-402c-966c-217a767f9bf3 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:40.716374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-01e57a72-4de8-402c-966c-217a767f9bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:40.800920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 482/976] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:40.825639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fa96b-26bc-4315-8bf7-2cd8f40c97d2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:40.828617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:40.938620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69fa96b-26bc-4315-8bf7-2cd8f40c97d2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:40.976003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d07f9-ff4b-4012-b2c6-c236cdd70815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:40.976674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d07f9-ff4b-4012-b2c6-c236cdd70815 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:40.976958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d07f9-ff4b-4012-b2c6-c236cdd70815 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:40.977234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d07f9-ff4b-4012-b2c6-c236cdd70815 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:40.978519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d07f9-ff4b-4012-b2c6-c236cdd70815 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:40.979995 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d07f9-ff4b-4012-b2c6-c236cdd70815 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:41.002715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 495/977] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:13:40 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:41.132114 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40f01f-42dd-4ac5-9e45-92ea525621da None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:41.132381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40f01f-42dd-4ac5-9e45-92ea525621da None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:41.132723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40f01f-42dd-4ac5-9e45-92ea525621da None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:41.134991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 483/978] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:41.140913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce18069-1a78-494e-b086-53ebd144426e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:41.141220 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce18069-1a78-494e-b086-53ebd144426e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:41.141472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce18069-1a78-494e-b086-53ebd144426e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:41.159916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8ce18069-1a78-494e-b086-53ebd144426e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:41.161124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8ce18069-1a78-494e-b086-53ebd144426e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:41.244422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:41.265665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 496/979] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:41.291133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40f01f-42dd-4ac5-9e45-92ea525621da None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:41.337467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:41.354123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bbc0f-83a9-45a9-b3da-162ed36f5b74 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:41.354415 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bbc0f-83a9-45a9-b3da-162ed36f5b74 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:41.354722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bbc0f-83a9-45a9-b3da-162ed36f5b74 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:41.356482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 497/980] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:41.361343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6326f6e-c5db-4412-8379-cc061097a808 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:41.361630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6326f6e-c5db-4412-8379-cc061097a808 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:41.361909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6326f6e-c5db-4412-8379-cc061097a808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:41.381868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a6326f6e-c5db-4412-8379-cc061097a808 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:41.383134 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a6326f6e-c5db-4412-8379-cc061097a808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:41.418866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40f01f-42dd-4ac5-9e45-92ea525621da None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:41.454762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697691b9-f623-4fc7-ba1b-75983b656094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:41.455580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697691b9-f623-4fc7-ba1b-75983b656094 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:41.455980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697691b9-f623-4fc7-ba1b-75983b656094 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:41.456351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697691b9-f623-4fc7-ba1b-75983b656094 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:41.457395 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697691b9-f623-4fc7-ba1b-75983b656094 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:41.459018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697691b9-f623-4fc7-ba1b-75983b656094 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:41.459974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 498/981] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:41.481645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.sql.core [None req-697691b9-f623-4fc7-ba1b-75983b656094 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 03 15:13:41.481645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 03 15:13:41.481645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [parameters: {'user_id': 'd5a24dd471924cc28eaccc8b3c40a35c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 03 15:13:41.481645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80221) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 03 15:13:41.483515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6326f6e-c5db-4412-8379-cc061097a808 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-697691b9-f623-4fc7-ba1b-75983b656094 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application conn.query(q) Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result.read() Jun 03 15:13:41.495215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 15:13:41.496198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application next(self.gen) Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application yield resource Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application next(self.gen) Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.session.rollback() Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.force_reraise() Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise self.value Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application session.commit() Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 03 15:13:41.497157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._prepare_impl() Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.session.flush() Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._flush(objects) Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application compat.raise_( Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application flush_context.execute() Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rec.execute(self) Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 03 15:13:41.498519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ Jun 03 15:13:41.499480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result = self._query(query) Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application conn.query(q) Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application result.read() Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 03 15:13:41.500353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd5a24dd471924cc28eaccc8b3c40a35c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:41.502516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 03 15:13:41.504330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 03 15:13:41.506342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 03 15:13:41.506342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 03 15:13:41.506342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 03 15:13:41.506342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 03 15:13:41.506342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 03 15:13:41.506342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:41.506342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 484/982] 10.222.0.93 () {60 vars in 1131 bytes} [Fri Jun 3 15:13:41 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:41.506342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697691b9-f623-4fc7-ba1b-75983b656094 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:41.645530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6326f6e-c5db-4412-8379-cc061097a808 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:41.652761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697691b9-f623-4fc7-ba1b-75983b656094 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:41.655053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:41.669958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:41.693681 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5954fd3-c211-48c7-b360-a01a69cd8164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:41.694467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5954fd3-c211-48c7-b360-a01a69cd8164 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:41.694467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5954fd3-c211-48c7-b360-a01a69cd8164 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:41.694963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5954fd3-c211-48c7-b360-a01a69cd8164 None admin] PATH_INFO: `/v3/services/image` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:41.695472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5954fd3-c211-48c7-b360-a01a69cd8164 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:41.697082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5954fd3-c211-48c7-b360-a01a69cd8164 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:41.714045 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c63ac4-9f35-4ec6-b7db-00becc81ad13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f5954fd3-c211-48c7-b360-a01a69cd8164 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:41.714847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:41.715867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 15:13:41.716735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:41.717544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c63ac4-9f35-4ec6-b7db-00becc81ad13 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:41.717544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c63ac4-9f35-4ec6-b7db-00becc81ad13 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:41.717544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c63ac4-9f35-4ec6-b7db-00becc81ad13 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:41.717737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 499/983] 10.222.0.93 () {56 vars in 1105 bytes} [Fri Jun 3 15:13:41 2022] GET /identity/v3/services/image => generated 86 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:41.717813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255fd15-9cbc-4c08-b90b-6ebaeeb6ba98 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:41.718488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255fd15-9cbc-4c08-b90b-6ebaeeb6ba98 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:41.718488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255fd15-9cbc-4c08-b90b-6ebaeeb6ba98 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:41.719809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 500/984] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:41.722686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255fd15-9cbc-4c08-b90b-6ebaeeb6ba98 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:41.731762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c63ac4-9f35-4ec6-b7db-00becc81ad13 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:41.734067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c63ac4-9f35-4ec6-b7db-00becc81ad13 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-55c63ac4-9f35-4ec6-b7db-00becc81ad13 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:41.753633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:41.755576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:41.757666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:41.757666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:41.757666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:41.757666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:41.757666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:41.757666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:41.757666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:41.757666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:41.757666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 03 15:13:41.757666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:41.760954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 485/985] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:13:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:41.761269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df5c82-e9cb-4d1e-8d9b-33687a550ec9 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:41.761557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df5c82-e9cb-4d1e-8d9b-33687a550ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:41.761830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df5c82-e9cb-4d1e-8d9b-33687a550ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:41.786072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-13df5c82-e9cb-4d1e-8d9b-33687a550ec9 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:41.787903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-13df5c82-e9cb-4d1e-8d9b-33687a550ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:41.876674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255fd15-9cbc-4c08-b90b-6ebaeeb6ba98 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:41.894590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 486/986] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:41.894590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df5c82-e9cb-4d1e-8d9b-33687a550ec9 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:41.951164 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5c065-2a99-4a71-9f0d-3a5d30c60d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:41.952418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5c065-2a99-4a71-9f0d-3a5d30c60d30 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:41.953025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5c065-2a99-4a71-9f0d-3a5d30c60d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:41.953584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5c065-2a99-4a71-9f0d-3a5d30c60d30 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:41.954433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5c065-2a99-4a71-9f0d-3a5d30c60d30 None admin] RBAC: Adding query filter params (name=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:41.954873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5c065-2a99-4a71-9f0d-3a5d30c60d30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:41.957207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5c065-2a99-4a71-9f0d-3a5d30c60d30 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:41.965230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 501/987] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:41.966240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5c065-2a99-4a71-9f0d-3a5d30c60d30 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:42.034263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:42.039557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:42.055486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df5c82-e9cb-4d1e-8d9b-33687a550ec9 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:42.103911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f97386-3ab6-49d3-950a-b0ba4f2d2619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:42.104846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f97386-3ab6-49d3-950a-b0ba4f2d2619 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:42.105304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f97386-3ab6-49d3-950a-b0ba4f2d2619 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:42.105789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f97386-3ab6-49d3-950a-b0ba4f2d2619 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:42.107165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f97386-3ab6-49d3-950a-b0ba4f2d2619 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:42.107677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f97386-3ab6-49d3-950a-b0ba4f2d2619 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:42.109893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f97386-3ab6-49d3-950a-b0ba4f2d2619 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:42.116568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5c065-2a99-4a71-9f0d-3a5d30c60d30 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:42.175767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef870877-0a2b-4e6c-9fa5-0e85f243833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:42.177158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef870877-0a2b-4e6c-9fa5-0e85f243833f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:42.184299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 487/988] 10.222.0.93 () {56 vars in 1101 bytes} [Fri Jun 3 15:13:41 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:42.184632 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef870877-0a2b-4e6c-9fa5-0e85f243833f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:42.184632 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef870877-0a2b-4e6c-9fa5-0e85f243833f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:42.186376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f97386-3ab6-49d3-950a-b0ba4f2d2619 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:42.198841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef870877-0a2b-4e6c-9fa5-0e85f243833f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:42.198841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef870877-0a2b-4e6c-9fa5-0e85f243833f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ef870877-0a2b-4e6c-9fa5-0e85f243833f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:42.222018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:42.224186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:42.226283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 502/989] 10.222.0.93 () {56 vars in 1126 bytes} [Fri Jun 3 15:13:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:42.231192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef870877-0a2b-4e6c-9fa5-0e85f243833f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:42.373356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef870877-0a2b-4e6c-9fa5-0e85f243833f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:42.394481 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:42.410689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f97386-3ab6-49d3-950a-b0ba4f2d2619 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:42.437909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422a974-087e-4b64-9d11-2d8bbf747f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:42.438694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422a974-087e-4b64-9d11-2d8bbf747f8e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:42.439160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422a974-087e-4b64-9d11-2d8bbf747f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:42.439605 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422a974-087e-4b64-9d11-2d8bbf747f8e None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:42.441120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422a974-087e-4b64-9d11-2d8bbf747f8e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:42.441622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422a974-087e-4b64-9d11-2d8bbf747f8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:42.444184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422a974-087e-4b64-9d11-2d8bbf747f8e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:42.457768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215ce8-b2a3-4d2a-9d4a-9d80b7d473d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:42.459056 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 503/990] 10.222.0.93 () {56 vars in 1122 bytes} [Fri Jun 3 15:13:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:42.459714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215ce8-b2a3-4d2a-9d4a-9d80b7d473d2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:42.460393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215ce8-b2a3-4d2a-9d4a-9d80b7d473d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:42.461147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88215ce8-b2a3-4d2a-9d4a-9d80b7d473d2 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:42.463471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88215ce8-b2a3-4d2a-9d4a-9d80b7d473d2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:42.463838 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422a974-087e-4b64-9d11-2d8bbf747f8e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:42.464666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88215ce8-b2a3-4d2a-9d4a-9d80b7d473d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:42.467070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88215ce8-b2a3-4d2a-9d4a-9d80b7d473d2 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:42.473311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 488/991] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:42 2022] GET /identity/v3/services?type=image => generated 122 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:42.636848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422a974-087e-4b64-9d11-2d8bbf747f8e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:42.690813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbca07-77e0-4b46-a388-6ffe1596a39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:42.691497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbca07-77e0-4b46-a388-6ffe1596a39e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:42.691770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbca07-77e0-4b46-a388-6ffe1596a39e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:42.692041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbca07-77e0-4b46-a388-6ffe1596a39e None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:42.692765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bbca07-77e0-4b46-a388-6ffe1596a39e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:42.693066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bbca07-77e0-4b46-a388-6ffe1596a39e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:42.694483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bbca07-77e0-4b46-a388-6ffe1596a39e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:42.701512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 504/992] 10.222.0.93 () {56 vars in 1122 bytes} [Fri Jun 3 15:13:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:42.707338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215ce8-b2a3-4d2a-9d4a-9d80b7d473d2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:42.727060 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:42.860246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88215ce8-b2a3-4d2a-9d4a-9d80b7d473d2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:42.909375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbb70b-c94c-4708-b0ff-21a26e0d4ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:42.909967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbb70b-c94c-4708-b0ff-21a26e0d4ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:42.910321 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbb70b-c94c-4708-b0ff-21a26e0d4ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:42.910775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbb70b-c94c-4708-b0ff-21a26e0d4ba4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:42.911592 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbb70b-c94c-4708-b0ff-21a26e0d4ba4 None admin] RBAC: Adding query filter params (interface=admin, service_id=d70029901b1d45d68e701868a5ac4cb3, region_id=RegionOne) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:42.912151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbb70b-c94c-4708-b0ff-21a26e0d4ba4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:42.913541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbb70b-c94c-4708-b0ff-21a26e0d4ba4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:42.922197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 489/993] 10.222.0.93 () {56 vars in 1249 bytes} [Fri Jun 3 15:13:42 2022] GET /identity/v3/endpoints?service_id=d70029901b1d45d68e701868a5ac4cb3&interface=admin®ion_id=RegionOne => generated 193 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:42.927759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbca07-77e0-4b46-a388-6ffe1596a39e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:42.996334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:43.081338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbca07-77e0-4b46-a388-6ffe1596a39e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:43.122417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051138b-64e5-477a-ab42-265c50cd32e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:43.123112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051138b-64e5-477a-ab42-265c50cd32e3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:43.123389 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051138b-64e5-477a-ab42-265c50cd32e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:43.123857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051138b-64e5-477a-ab42-265c50cd32e3 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:43.124400 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051138b-64e5-477a-ab42-265c50cd32e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:43.125973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051138b-64e5-477a-ab42-265c50cd32e3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:43.134963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 505/994] 10.222.0.93 () {56 vars in 1087 bytes} [Fri Jun 3 15:13:42 2022] GET /identity/v3/services => generated 1727 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:13:43.504604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868834f-98b4-42c5-b60b-bea3bb5a8c56 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:43.505121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868834f-98b4-42c5-b60b-bea3bb5a8c56 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:43.505510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868834f-98b4-42c5-b60b-bea3bb5a8c56 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:43.507472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 490/995] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:43.513207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d9f86-bb39-4c56-8bdf-663dd449b06c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:43.513497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d9f86-bb39-4c56-8bdf-663dd449b06c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:43.513776 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d9f86-bb39-4c56-8bdf-663dd449b06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:43.532797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-208d9f86-bb39-4c56-8bdf-663dd449b06c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:43.534266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-208d9f86-bb39-4c56-8bdf-663dd449b06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:43.621674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 506/996] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:43.648525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868834f-98b4-42c5-b60b-bea3bb5a8c56 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:43.786113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:43.811265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868834f-98b4-42c5-b60b-bea3bb5a8c56 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:43.846974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18426364-40d5-4b74-9a60-ea4ba23407c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:43.847774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18426364-40d5-4b74-9a60-ea4ba23407c7 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:43.848057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18426364-40d5-4b74-9a60-ea4ba23407c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:43.848334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18426364-40d5-4b74-9a60-ea4ba23407c7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:43.848998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18426364-40d5-4b74-9a60-ea4ba23407c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:43.850560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18426364-40d5-4b74-9a60-ea4ba23407c7 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-18426364-40d5-4b74-9a60-ea4ba23407c7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:43.862655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:43.863680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:43.865126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 491/997] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:43.870037 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d9f86-bb39-4c56-8bdf-663dd449b06c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:43.910232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:44.002217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d9f86-bb39-4c56-8bdf-663dd449b06c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:44.038019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511cbf8-0990-4b3b-8f74-ada47b3497e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:44.038662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511cbf8-0990-4b3b-8f74-ada47b3497e7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:44.038934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511cbf8-0990-4b3b-8f74-ada47b3497e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:44.039195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511cbf8-0990-4b3b-8f74-ada47b3497e7 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:44.039898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511cbf8-0990-4b3b-8f74-ada47b3497e7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:44.040726 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511cbf8-0990-4b3b-8f74-ada47b3497e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:44.042098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511cbf8-0990-4b3b-8f74-ada47b3497e7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:44.065761 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 507/998] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:44.067314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18426364-40d5-4b74-9a60-ea4ba23407c7 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:44.189795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18426364-40d5-4b74-9a60-ea4ba23407c7 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:44.243935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383ac37-476b-4cf8-8f68-49d2e32f1fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:44.244611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383ac37-476b-4cf8-8f68-49d2e32f1fde None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:44.244899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383ac37-476b-4cf8-8f68-49d2e32f1fde None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:44.245159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383ac37-476b-4cf8-8f68-49d2e32f1fde None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:44.245881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383ac37-476b-4cf8-8f68-49d2e32f1fde None admin] RBAC: Adding query filter params (type=compute) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:44.246193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383ac37-476b-4cf8-8f68-49d2e32f1fde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:44.247631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383ac37-476b-4cf8-8f68-49d2e32f1fde None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:44.253323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 492/999] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:44.259772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511cbf8-0990-4b3b-8f74-ada47b3497e7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:44.418576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:44.427815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511cbf8-0990-4b3b-8f74-ada47b3497e7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:44.466505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac32ad-2b74-47d4-96ca-7aaa97d7ff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:44.467213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac32ad-2b74-47d4-96ca-7aaa97d7ff8e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:44.467498 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac32ad-2b74-47d4-96ca-7aaa97d7ff8e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:44.467781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac32ad-2b74-47d4-96ca-7aaa97d7ff8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:44.468587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac32ad-2b74-47d4-96ca-7aaa97d7ff8e None admin] RBAC: Adding query filter params (interface=public, service_id=54ef00f84fb848faabe0b083eb781bbf, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:44.468903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac32ad-2b74-47d4-96ca-7aaa97d7ff8e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:44.470307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac32ad-2b74-47d4-96ca-7aaa97d7ff8e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:44.482575 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 508/1000] 10.222.0.93 () {56 vars in 1251 bytes} [Fri Jun 3 15:13:44 2022] GET /identity/v3/endpoints?service_id=54ef00f84fb848faabe0b083eb781bbf&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:44.483962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383ac37-476b-4cf8-8f68-49d2e32f1fde None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:44.495696 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:44.619696 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383ac37-476b-4cf8-8f68-49d2e32f1fde None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:44.664692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54307ab8-f4be-4548-aab3-807b5aa0a835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:44.665811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54307ab8-f4be-4548-aab3-807b5aa0a835 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:44.666421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54307ab8-f4be-4548-aab3-807b5aa0a835 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:44.667014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54307ab8-f4be-4548-aab3-807b5aa0a835 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:44.668295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54307ab8-f4be-4548-aab3-807b5aa0a835 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:44.673044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54307ab8-f4be-4548-aab3-807b5aa0a835 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:44.680121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 493/1001] 10.222.0.93 () {56 vars in 1087 bytes} [Fri Jun 3 15:13:44 2022] GET /identity/v3/services => generated 1727 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:13:45.226197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c787749-1d53-4cbe-ac02-c47ba4c991d2 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:45.226789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c787749-1d53-4cbe-ac02-c47ba4c991d2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:45.226789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c787749-1d53-4cbe-ac02-c47ba4c991d2 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:45.229341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 509/1002] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:45.233789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6ba58-84f7-4ab2-b52e-ce0ca60393a7 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:45.234230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6ba58-84f7-4ab2-b52e-ce0ca60393a7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:45.234323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6ba58-84f7-4ab2-b52e-ce0ca60393a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:45.251582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e4c6ba58-84f7-4ab2-b52e-ce0ca60393a7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:45.252853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e4c6ba58-84f7-4ab2-b52e-ce0ca60393a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:45.321699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:45.334715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 494/1003] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:45.350408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c787749-1d53-4cbe-ac02-c47ba4c991d2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:45.463961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c787749-1d53-4cbe-ac02-c47ba4c991d2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:45.489182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68b4bb-fa46-4f09-a555-43aa02e97a5f None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:45.489366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68b4bb-fa46-4f09-a555-43aa02e97a5f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:45.489641 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68b4bb-fa46-4f09-a555-43aa02e97a5f None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:45.491239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 495/1004] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:45.496412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38389022-af16-4069-89c5-e0c70b288f08 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:45.496904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38389022-af16-4069-89c5-e0c70b288f08 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:45.497014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38389022-af16-4069-89c5-e0c70b288f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:45.504320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809b7a3-7744-4a98-8d58-ce3fef3bba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:45.504986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809b7a3-7744-4a98-8d58-ce3fef3bba92 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:45.505268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809b7a3-7744-4a98-8d58-ce3fef3bba92 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:45.505542 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3809b7a3-7744-4a98-8d58-ce3fef3bba92 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:45.506273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809b7a3-7744-4a98-8d58-ce3fef3bba92 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:45.507678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3809b7a3-7744-4a98-8d58-ce3fef3bba92 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:45.514844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-38389022-af16-4069-89c5-e0c70b288f08 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:45.516245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-38389022-af16-4069-89c5-e0c70b288f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:45.526570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 510/1005] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:13:45 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:13:45.606605 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 496/1006] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:45 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:45.629811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809b7a3-7744-4a98-8d58-ce3fef3bba92 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:45.660199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:45.752764 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3809b7a3-7744-4a98-8d58-ce3fef3bba92 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:45.791098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fdb3f-e42d-4ff3-b225-0b5b9c752408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:45.791098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fdb3f-e42d-4ff3-b225-0b5b9c752408 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:45.791098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fdb3f-e42d-4ff3-b225-0b5b9c752408 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:45.791430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fdb3f-e42d-4ff3-b225-0b5b9c752408 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:45.791916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fdb3f-e42d-4ff3-b225-0b5b9c752408 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:45.793292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fdb3f-e42d-4ff3-b225-0b5b9c752408 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-170fdb3f-e42d-4ff3-b225-0b5b9c752408 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:45.806500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:45.807602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:45.808451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 511/1007] 10.222.0.93 () {56 vars in 1126 bytes} [Fri Jun 3 15:13:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:45.815841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38389022-af16-4069-89c5-e0c70b288f08 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:45.937384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38389022-af16-4069-89c5-e0c70b288f08 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:45.967606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:45.982496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4fdb0-25aa-45c4-8056-5c1229b736f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:45.983191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4fdb0-25aa-45c4-8056-5c1229b736f9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:45.983464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4fdb0-25aa-45c4-8056-5c1229b736f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:45.983733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4fdb0-25aa-45c4-8056-5c1229b736f9 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:45.984443 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4fdb0-25aa-45c4-8056-5c1229b736f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:45.984746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4fdb0-25aa-45c4-8056-5c1229b736f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:45.986124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4fdb0-25aa-45c4-8056-5c1229b736f9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:45.991719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 497/1008] 10.222.0.93 () {56 vars in 1122 bytes} [Fri Jun 3 15:13:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:45.997908 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fdb3f-e42d-4ff3-b225-0b5b9c752408 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:46.089187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:46.109175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fdb3f-e42d-4ff3-b225-0b5b9c752408 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:46.143294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dac3c-95bb-4078-b829-c76f6d61db43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:46.143956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dac3c-95bb-4078-b829-c76f6d61db43 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:46.144244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dac3c-95bb-4078-b829-c76f6d61db43 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:46.144516 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dac3c-95bb-4078-b829-c76f6d61db43 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:46.145296 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dac3c-95bb-4078-b829-c76f6d61db43 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:46.145604 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dac3c-95bb-4078-b829-c76f6d61db43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:46.147059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dac3c-95bb-4078-b829-c76f6d61db43 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:46.159071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 512/1009] 10.222.0.93 () {56 vars in 1122 bytes} [Fri Jun 3 15:13:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:46.163157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4fdb0-25aa-45c4-8056-5c1229b736f9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:46.278655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4fdb0-25aa-45c4-8056-5c1229b736f9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:46.313750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0fe38-4487-4ad8-ad0e-f56316babddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:46.314420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0fe38-4487-4ad8-ad0e-f56316babddf None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:46.314727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0fe38-4487-4ad8-ad0e-f56316babddf None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:46.315007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0fe38-4487-4ad8-ad0e-f56316babddf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:46.315763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0fe38-4487-4ad8-ad0e-f56316babddf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:46.317135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0fe38-4487-4ad8-ad0e-f56316babddf None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:46.342311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 498/1010] 10.222.0.93 () {60 vars in 1144 bytes} [Fri Jun 3 15:13:46 2022] POST /identity/v3/endpoints => generated 325 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:47.010966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43273a2e-f378-4b0b-97ee-a1664f52cf8e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:47.012206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43273a2e-f378-4b0b-97ee-a1664f52cf8e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:47.012206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43273a2e-f378-4b0b-97ee-a1664f52cf8e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:47.013216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 513/1011] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:47.018184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5500de-4abf-45d6-b810-5d15922407da None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:47.018477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5500de-4abf-45d6-b810-5d15922407da None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:47.018780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5500de-4abf-45d6-b810-5d15922407da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:47.036948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-7e5500de-4abf-45d6-b810-5d15922407da None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:47.038124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7e5500de-4abf-45d6-b810-5d15922407da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:47.087217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:47.115550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 499/1012] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:47 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:47.139683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43273a2e-f378-4b0b-97ee-a1664f52cf8e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:47.256606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43273a2e-f378-4b0b-97ee-a1664f52cf8e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:47.293448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae17a0-5589-4d55-8901-083aa0120e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:47.294155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae17a0-5589-4d55-8901-083aa0120e5d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:47.294433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae17a0-5589-4d55-8901-083aa0120e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:47.294737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae17a0-5589-4d55-8901-083aa0120e5d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:47.295425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae17a0-5589-4d55-8901-083aa0120e5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:47.296767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae17a0-5589-4d55-8901-083aa0120e5d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-78ae17a0-5589-4d55-8901-083aa0120e5d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:47.309661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:47.310804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 03 15:13:47.311677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:47.313475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 514/1013] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:47.317058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5500de-4abf-45d6-b810-5d15922407da None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:47.427959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5500de-4abf-45d6-b810-5d15922407da None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:47.474580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1679b07-085a-42a3-8f22-5cc3523055c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:47.475790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1679b07-085a-42a3-8f22-5cc3523055c1 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:47.476279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1679b07-085a-42a3-8f22-5cc3523055c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:47.476747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1679b07-085a-42a3-8f22-5cc3523055c1 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:47.477702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1679b07-085a-42a3-8f22-5cc3523055c1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:47.478193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1679b07-085a-42a3-8f22-5cc3523055c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:47.479896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1679b07-085a-42a3-8f22-5cc3523055c1 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:47.490024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 500/1014] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:47.491240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae17a0-5589-4d55-8901-083aa0120e5d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:47.508497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:47.603260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae17a0-5589-4d55-8901-083aa0120e5d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:47.639479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfa5fa-252d-49f6-a087-8bab801f2ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:47.640144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfa5fa-252d-49f6-a087-8bab801f2ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:47.640425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfa5fa-252d-49f6-a087-8bab801f2ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:47.640697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfa5fa-252d-49f6-a087-8bab801f2ba4 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:47.641400 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfa5fa-252d-49f6-a087-8bab801f2ba4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:47.641704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfa5fa-252d-49f6-a087-8bab801f2ba4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:47.643193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfa5fa-252d-49f6-a087-8bab801f2ba4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:47.648483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 515/1015] 10.222.0.93 () {56 vars in 1112 bytes} [Fri Jun 3 15:13:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:47.656351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1679b07-085a-42a3-8f22-5cc3523055c1 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:47.705028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:47.769553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1679b07-085a-42a3-8f22-5cc3523055c1 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:47.805940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05caf-5661-4df0-b06b-828d33449525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:47.806665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05caf-5661-4df0-b06b-828d33449525 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:47.806950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05caf-5661-4df0-b06b-828d33449525 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:47.807269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05caf-5661-4df0-b06b-828d33449525 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:47.808020 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc05caf-5661-4df0-b06b-828d33449525 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:47.809404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc05caf-5661-4df0-b06b-828d33449525 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:47.828672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 501/1016] 10.222.0.93 () {60 vars in 1144 bytes} [Fri Jun 3 15:13:47 2022] POST /identity/v3/endpoints => generated 334 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:47.865866 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280da8a9-32ec-49bc-a68c-a926562c73b7 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:47.866224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280da8a9-32ec-49bc-a68c-a926562c73b7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:47.866658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280da8a9-32ec-49bc-a68c-a926562c73b7 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:47.868758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 516/1017] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:47.874876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871099b-99c4-44bb-a763-7ebedfc5f955 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:47.875196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871099b-99c4-44bb-a763-7ebedfc5f955 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:47.875454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871099b-99c4-44bb-a763-7ebedfc5f955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:47.893697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-7871099b-99c4-44bb-a763-7ebedfc5f955 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:47.894982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7871099b-99c4-44bb-a763-7ebedfc5f955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:47.995343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 502/1018] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:48.018630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280da8a9-32ec-49bc-a68c-a926562c73b7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:48.113474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:48.142068 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280da8a9-32ec-49bc-a68c-a926562c73b7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:48.181905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17090647-36ea-4ccb-9546-a7ed37908464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:48.182732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17090647-36ea-4ccb-9546-a7ed37908464 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:48.183014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17090647-36ea-4ccb-9546-a7ed37908464 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:48.183292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17090647-36ea-4ccb-9546-a7ed37908464 None admin] PATH_INFO: `/v3/services/image` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:48.183949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17090647-36ea-4ccb-9546-a7ed37908464 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:48.185319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17090647-36ea-4ccb-9546-a7ed37908464 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-17090647-36ea-4ccb-9546-a7ed37908464 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:48.197655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:48.199634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:48.200990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 517/1019] 10.222.0.93 () {56 vars in 1105 bytes} [Fri Jun 3 15:13:48 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:48.205229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871099b-99c4-44bb-a763-7ebedfc5f955 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:48.233217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:48.318715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871099b-99c4-44bb-a763-7ebedfc5f955 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:48.355836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71f699-1500-4410-a61c-9cc3892949b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:48.356562 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71f699-1500-4410-a61c-9cc3892949b9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:48.356846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71f699-1500-4410-a61c-9cc3892949b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:48.357120 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71f699-1500-4410-a61c-9cc3892949b9 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:48.357824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71f699-1500-4410-a61c-9cc3892949b9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:48.358129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71f699-1500-4410-a61c-9cc3892949b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:48.359547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71f699-1500-4410-a61c-9cc3892949b9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:48.364820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 503/1020] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:48.371159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17090647-36ea-4ccb-9546-a7ed37908464 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:48.480630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17090647-36ea-4ccb-9546-a7ed37908464 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:48.515457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216d947-123e-4ea9-b401-0f25786bfbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:48.516146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216d947-123e-4ea9-b401-0f25786bfbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:48.516411 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216d947-123e-4ea9-b401-0f25786bfbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:48.516687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216d947-123e-4ea9-b401-0f25786bfbe9 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:48.517391 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216d947-123e-4ea9-b401-0f25786bfbe9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:48.517697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216d947-123e-4ea9-b401-0f25786bfbe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:48.519117 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216d947-123e-4ea9-b401-0f25786bfbe9 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:48.524473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 518/1021] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:48.530090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71f699-1500-4410-a61c-9cc3892949b9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:48.632406 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca60dd5-2f2a-4cfa-8ef5-6af582bb6b1b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:48.632774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca60dd5-2f2a-4cfa-8ef5-6af582bb6b1b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:48.633405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca60dd5-2f2a-4cfa-8ef5-6af582bb6b1b None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:48.634638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 519/1022] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:48.639997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c47922-e0bb-4cb1-82e0-4a7165403774 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:48.640344 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c47922-e0bb-4cb1-82e0-4a7165403774 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:48.640568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c47922-e0bb-4cb1-82e0-4a7165403774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:48.641593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:48.646573 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:48.659332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f4c47922-e0bb-4cb1-82e0-4a7165403774 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:48.660613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f4c47922-e0bb-4cb1-82e0-4a7165403774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:48.663998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71f699-1500-4410-a61c-9cc3892949b9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:48.702271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e980a-975b-44cb-aa98-6d84955331ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:48.702971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e980a-975b-44cb-aa98-6d84955331ee None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:48.703249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e980a-975b-44cb-aa98-6d84955331ee None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:48.703520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e980a-975b-44cb-aa98-6d84955331ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:48.704316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e980a-975b-44cb-aa98-6d84955331ee None admin] RBAC: Adding query filter params (interface=public, service_id=24c2dd56d408405c9d66f4f05e6f2f29, region_id=RegionOne) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:48.704620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e980a-975b-44cb-aa98-6d84955331ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:48.705997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e980a-975b-44cb-aa98-6d84955331ee None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:48.711428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 504/1023] 10.222.0.93 () {56 vars in 1251 bytes} [Fri Jun 3 15:13:48 2022] GET /identity/v3/endpoints?service_id=24c2dd56d408405c9d66f4f05e6f2f29&interface=public®ion_id=RegionOne => generated 194 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:48.718329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e980a-975b-44cb-aa98-6d84955331ee None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:48.743903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 520/1024] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:48.767408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c47922-e0bb-4cb1-82e0-4a7165403774 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:48.851058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e980a-975b-44cb-aa98-6d84955331ee None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:48.891092 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c47922-e0bb-4cb1-82e0-4a7165403774 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:48.902289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59f626-e445-49c1-a8a9-1f6f2075060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:48.902289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59f626-e445-49c1-a8a9-1f6f2075060a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:48.902289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59f626-e445-49c1-a8a9-1f6f2075060a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:48.902289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59f626-e445-49c1-a8a9-1f6f2075060a None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:48.902853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59f626-e445-49c1-a8a9-1f6f2075060a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:48.902853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59f626-e445-49c1-a8a9-1f6f2075060a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:48.911120 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 505/1025] 10.222.0.93 () {56 vars in 1087 bytes} [Fri Jun 3 15:13:48 2022] GET /identity/v3/services => generated 1957 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:13:48.946561 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0699a05-6776-441e-9744-62ca6df6db46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:48.947365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0699a05-6776-441e-9744-62ca6df6db46 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:48.947647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0699a05-6776-441e-9744-62ca6df6db46 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:48.947914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0699a05-6776-441e-9744-62ca6df6db46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:48.949229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0699a05-6776-441e-9744-62ca6df6db46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:48.950729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0699a05-6776-441e-9744-62ca6df6db46 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:48.953186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 521/1026] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:48.963205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59f626-e445-49c1-a8a9-1f6f2075060a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:48.983171 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:49.095068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59f626-e445-49c1-a8a9-1f6f2075060a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:49.134189 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066827b-78b1-4f59-8d28-94e4a9c4bd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:49.134949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066827b-78b1-4f59-8d28-94e4a9c4bd0b None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:49.135246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066827b-78b1-4f59-8d28-94e4a9c4bd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:49.135521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066827b-78b1-4f59-8d28-94e4a9c4bd0b None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:49.136245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066827b-78b1-4f59-8d28-94e4a9c4bd0b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:49.138399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066827b-78b1-4f59-8d28-94e4a9c4bd0b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:49.168607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 506/1027] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:13:48 2022] POST /identity/v3/projects => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:50.350950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd6e9d-70e4-4fc2-b0ee-ec6f896f2713 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:50.351678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd6e9d-70e4-4fc2-b0ee-ec6f896f2713 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:50.351931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd6e9d-70e4-4fc2-b0ee-ec6f896f2713 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:50.353745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 522/1028] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:50.360060 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b164ca-d410-4acd-84b6-acb18afd4fe2 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:50.361019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b164ca-d410-4acd-84b6-acb18afd4fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:50.361019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b164ca-d410-4acd-84b6-acb18afd4fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:50.380853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c2b164ca-d410-4acd-84b6-acb18afd4fe2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:50.382058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c2b164ca-d410-4acd-84b6-acb18afd4fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:50.466221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:50.482702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 507/1029] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:50.507298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd6e9d-70e4-4fc2-b0ee-ec6f896f2713 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:50.510314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:50.637891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd6e9d-70e4-4fc2-b0ee-ec6f896f2713 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:50.681897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257899-aa92-4359-a8f9-4f96c6ce9b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:50.682760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257899-aa92-4359-a8f9-4f96c6ce9b4d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:50.683048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257899-aa92-4359-a8f9-4f96c6ce9b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:50.683324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257899-aa92-4359-a8f9-4f96c6ce9b4d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:50.694378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49257899-aa92-4359-a8f9-4f96c6ce9b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-49257899-aa92-4359-a8f9-4f96c6ce9b4d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:50.702664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:50.703618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:50.709504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 523/1030] 10.222.0.93 () {56 vars in 1120 bytes} [Fri Jun 3 15:13:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:50.712023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b164ca-d410-4acd-84b6-acb18afd4fe2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:50.820729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b164ca-d410-4acd-84b6-acb18afd4fe2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:50.868917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed97c8-5207-42c1-b89f-73a95b30ed04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:50.869615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed97c8-5207-42c1-b89f-73a95b30ed04 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:50.869899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed97c8-5207-42c1-b89f-73a95b30ed04 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:50.870174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed97c8-5207-42c1-b89f-73a95b30ed04 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:50.870969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed97c8-5207-42c1-b89f-73a95b30ed04 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:50.871275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed97c8-5207-42c1-b89f-73a95b30ed04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:50.872675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed97c8-5207-42c1-b89f-73a95b30ed04 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:50.881784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 508/1031] 10.222.0.93 () {56 vars in 1115 bytes} [Fri Jun 3 15:13:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:50.887619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257899-aa92-4359-a8f9-4f96c6ce9b4d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:50.955336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:51.009988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257899-aa92-4359-a8f9-4f96c6ce9b4d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:51.057402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a9243-afba-4f7c-9d91-f5888cf908e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:51.058216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a9243-afba-4f7c-9d91-f5888cf908e0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:51.058501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a9243-afba-4f7c-9d91-f5888cf908e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:51.058807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a9243-afba-4f7c-9d91-f5888cf908e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:51.071020 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a9243-afba-4f7c-9d91-f5888cf908e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:51.072602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a9243-afba-4f7c-9d91-f5888cf908e0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-d45a9243-afba-4f7c-9d91-f5888cf908e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:51.091003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:51.092091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:51.093022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:51.093022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:51.093022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:51.093022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:51.093022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 524/1032] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:51.097989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed97c8-5207-42c1-b89f-73a95b30ed04 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:51.163221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:51.213178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed97c8-5207-42c1-b89f-73a95b30ed04 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:51.266035 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdef1bb-4b3d-4dab-9c1d-61b5cebfbf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:51.266810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdef1bb-4b3d-4dab-9c1d-61b5cebfbf13 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:51.267097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdef1bb-4b3d-4dab-9c1d-61b5cebfbf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:51.267378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdef1bb-4b3d-4dab-9c1d-61b5cebfbf13 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:51.268140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdef1bb-4b3d-4dab-9c1d-61b5cebfbf13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:51.268454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdef1bb-4b3d-4dab-9c1d-61b5cebfbf13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:51.269928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdef1bb-4b3d-4dab-9c1d-61b5cebfbf13 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:51.284275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 509/1033] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:51.290151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a9243-afba-4f7c-9d91-f5888cf908e0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:51.372951 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578340-fc26-446e-912e-12185b0dc269 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:51.373596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578340-fc26-446e-912e-12185b0dc269 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:51.373596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc578340-fc26-446e-912e-12185b0dc269 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:51.374895 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 510/1034] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:13:51.379933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca0944-5ad4-4696-8f52-f853db42842d None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:51.380215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca0944-5ad4-4696-8f52-f853db42842d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:51.380493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca0944-5ad4-4696-8f52-f853db42842d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:51.397880 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d7ca0944-5ad4-4696-8f52-f853db42842d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:51.399814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d7ca0944-5ad4-4696-8f52-f853db42842d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:51.427534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a9243-afba-4f7c-9d91-f5888cf908e0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:51.443063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:51.468745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbc4eb-bc6d-48ec-9ae4-449dd93f4618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:51.469460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbc4eb-bc6d-48ec-9ae4-449dd93f4618 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:51.469737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbc4eb-bc6d-48ec-9ae4-449dd93f4618 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:51.470004 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bbc4eb-bc6d-48ec-9ae4-449dd93f4618 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:51.476383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 511/1035] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:51.480223 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bbc4eb-bc6d-48ec-9ae4-449dd93f4618 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:51.481897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bbc4eb-bc6d-48ec-9ae4-449dd93f4618 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-23bbc4eb-bc6d-48ec-9ae4-449dd93f4618 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:51.498557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:51.500404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:51.501419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:51.501419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:51.501419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:51.501419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:51.501419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:51.501419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:51.501419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:51.501419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:51.501419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:13:51.501419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:51.501998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca0944-5ad4-4696-8f52-f853db42842d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:51.502082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 525/1036] 10.222.0.93 () {56 vars in 1093 bytes} [Fri Jun 3 15:13:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:51.506858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbc4eb-bc6d-48ec-9ae4-449dd93f4618 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:51.564104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:51.637072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca0944-5ad4-4696-8f52-f853db42842d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:51.638810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bbc4eb-bc6d-48ec-9ae4-449dd93f4618 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:51.675701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f01bd0-d3c9-4064-bf33-1dc7c8ddbaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:51.676076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f01bd0-d3c9-4064-bf33-1dc7c8ddbaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:51.676366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f01bd0-d3c9-4064-bf33-1dc7c8ddbaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:51.676644 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f01bd0-d3c9-4064-bf33-1dc7c8ddbaf7 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:51.677587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f01bd0-d3c9-4064-bf33-1dc7c8ddbaf7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:51.677853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f01bd0-d3c9-4064-bf33-1dc7c8ddbaf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:51.678413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904ac91-a3f2-4caf-a454-e83a90eaf59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:51.679242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904ac91-a3f2-4caf-a454-e83a90eaf59e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:51.679352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f01bd0-d3c9-4064-bf33-1dc7c8ddbaf7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:51.679523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904ac91-a3f2-4caf-a454-e83a90eaf59e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:51.679801 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904ac91-a3f2-4caf-a454-e83a90eaf59e None admin] PATH_INFO: `/v3/services/image` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:51.680989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904ac91-a3f2-4caf-a454-e83a90eaf59e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:51.682361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904ac91-a3f2-4caf-a454-e83a90eaf59e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-7904ac91-a3f2-4caf-a454-e83a90eaf59e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:51.694327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:51.695524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:51.697307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 512/1037] 10.222.0.93 () {56 vars in 1105 bytes} [Fri Jun 3 15:13:51 2022] GET /identity/v3/services/image => generated 86 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:51.703287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c34bf2-7722-4d73-942f-0e6e47dcc0d5 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:51.703287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c34bf2-7722-4d73-942f-0e6e47dcc0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:51.703287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c34bf2-7722-4d73-942f-0e6e47dcc0d5 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:51.703651 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 526/1038] 10.222.0.93 () {56 vars in 1133 bytes} [Fri Jun 3 15:13:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:51.703752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 513/1039] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:51.703839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f01bd0-d3c9-4064-bf33-1dc7c8ddbaf7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:51.705581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c34bf2-7722-4d73-942f-0e6e47dcc0d5 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:51.735016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:51.807568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f01bd0-d3c9-4064-bf33-1dc7c8ddbaf7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:51.810804 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c34bf2-7722-4d73-942f-0e6e47dcc0d5 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:51.824121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:51.844095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e01b062-08a2-41e3-aabe-b0bf96dacc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:51.844766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01b062-08a2-41e3-aabe-b0bf96dacc62 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:51.845050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01b062-08a2-41e3-aabe-b0bf96dacc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:51.845326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e01b062-08a2-41e3-aabe-b0bf96dacc62 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:51.846033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01b062-08a2-41e3-aabe-b0bf96dacc62 None admin] RBAC: Adding query filter params (name=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:51.846150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333558ba-bd9d-482f-ac19-61cdc827046b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:51.846339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01b062-08a2-41e3-aabe-b0bf96dacc62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:51.846850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333558ba-bd9d-482f-ac19-61cdc827046b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:51.847157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333558ba-bd9d-482f-ac19-61cdc827046b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:51.847429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333558ba-bd9d-482f-ac19-61cdc827046b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:51.847753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e01b062-08a2-41e3-aabe-b0bf96dacc62 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:51.859194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 527/1040] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:51.859194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ede07b-29dd-4bd5-b91b-f1a288fb3806 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:51.859194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ede07b-29dd-4bd5-b91b-f1a288fb3806 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:51.859194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ede07b-29dd-4bd5-b91b-f1a288fb3806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:51.873991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b9ede07b-29dd-4bd5-b91b-f1a288fb3806 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:51.875098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b9ede07b-29dd-4bd5-b91b-f1a288fb3806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:51.922231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333558ba-bd9d-482f-ac19-61cdc827046b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:51.923735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333558ba-bd9d-482f-ac19-61cdc827046b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:51.948589 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 528/1041] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:51.949444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ede07b-29dd-4bd5-b91b-f1a288fb3806 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:51.978387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:52.042298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 514/1042] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:51 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:13:52.043199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333558ba-bd9d-482f-ac19-61cdc827046b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:52.065001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ede07b-29dd-4bd5-b91b-f1a288fb3806 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:52.065631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:52.099716 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bc54d-187e-4849-b184-3490fb491b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:52.100381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc54d-187e-4849-b184-3490fb491b79 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:52.100666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc54d-187e-4849-b184-3490fb491b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:52.100940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc54d-187e-4849-b184-3490fb491b79 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:52.101654 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bc54d-187e-4849-b184-3490fb491b79 None admin] RBAC: Adding query filter params (type=image) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:52.101965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bc54d-187e-4849-b184-3490fb491b79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:52.103898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bc54d-187e-4849-b184-3490fb491b79 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:52.108805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 529/1043] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:52.109599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bc54d-187e-4849-b184-3490fb491b79 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:52.159215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333558ba-bd9d-482f-ac19-61cdc827046b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:52.201575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b4349-c5ce-48f8-ac54-1e56f96ab316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:52.202371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b4349-c5ce-48f8-ac54-1e56f96ab316 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:52.202763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b4349-c5ce-48f8-ac54-1e56f96ab316 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:52.203046 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b4349-c5ce-48f8-ac54-1e56f96ab316 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:52.204242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b4349-c5ce-48f8-ac54-1e56f96ab316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:52.205679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b4349-c5ce-48f8-ac54-1e56f96ab316 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:52.206366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:52.207381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 515/1044] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:52.208165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b4349-c5ce-48f8-ac54-1e56f96ab316 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:52.218466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bc54d-187e-4849-b184-3490fb491b79 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:52.255162 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f2722-f6bc-41b2-bd14-6b733b5b1af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:52.255962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f2722-f6bc-41b2-bd14-6b733b5b1af5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:52.256239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f2722-f6bc-41b2-bd14-6b733b5b1af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:52.256512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57f2722-f6bc-41b2-bd14-6b733b5b1af5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:52.266724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f2722-f6bc-41b2-bd14-6b733b5b1af5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:52.268213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57f2722-f6bc-41b2-bd14-6b733b5b1af5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c57f2722-f6bc-41b2-bd14-6b733b5b1af5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:52.285398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:52.286387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:52.287621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:52.287621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:52.287621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:52.287621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:52.287621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 530/1045] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:52.287900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f2722-f6bc-41b2-bd14-6b733b5b1af5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:52.293372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:52.314300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b4349-c5ce-48f8-ac54-1e56f96ab316 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:52.349397 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b1c0e-74c4-495a-be1e-4db103050623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:52.350061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b1c0e-74c4-495a-be1e-4db103050623 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:52.350349 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b1c0e-74c4-495a-be1e-4db103050623 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:52.350672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b1c0e-74c4-495a-be1e-4db103050623 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:52.351403 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b1c0e-74c4-495a-be1e-4db103050623 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:52.352788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b1c0e-74c4-495a-be1e-4db103050623 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:52.371939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 516/1046] 10.222.0.93 () {60 vars in 1144 bytes} [Fri Jun 3 15:13:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:52.372762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b1c0e-74c4-495a-be1e-4db103050623 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:52.404967 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57f2722-f6bc-41b2-bd14-6b733b5b1af5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:52.462458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b285961-2fe5-43d1-b643-ad6f98d0d330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:52.463688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b285961-2fe5-43d1-b643-ad6f98d0d330 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:52.464175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b285961-2fe5-43d1-b643-ad6f98d0d330 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:52.464652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b285961-2fe5-43d1-b643-ad6f98d0d330 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:52.465956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b285961-2fe5-43d1-b643-ad6f98d0d330 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:52.468749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b285961-2fe5-43d1-b643-ad6f98d0d330 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:52.483388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:52.497141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b1c0e-74c4-495a-be1e-4db103050623 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:52.502029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 531/1047] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:13:52 2022] POST /identity/v3/users => generated 285 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:13:52.548478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a594a5-d494-462c-85b8-3db6e3fcbe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:52.549493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a594a5-d494-462c-85b8-3db6e3fcbe28 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:52.549940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a594a5-d494-462c-85b8-3db6e3fcbe28 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:52.550370 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a594a5-d494-462c-85b8-3db6e3fcbe28 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:52.551323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a594a5-d494-462c-85b8-3db6e3fcbe28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:52.551819 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a594a5-d494-462c-85b8-3db6e3fcbe28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:52.553649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a594a5-d494-462c-85b8-3db6e3fcbe28 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:52.557145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:52.579241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 517/1048] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:52.584848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b285961-2fe5-43d1-b643-ad6f98d0d330 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:52.729963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b285961-2fe5-43d1-b643-ad6f98d0d330 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:52.794067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3231a40-345f-454c-8e23-77a427d9f850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:52.795477 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3231a40-345f-454c-8e23-77a427d9f850 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:52.795948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3231a40-345f-454c-8e23-77a427d9f850 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:52.796426 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3231a40-345f-454c-8e23-77a427d9f850 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:52.809091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3231a40-345f-454c-8e23-77a427d9f850 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:52.811831 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3231a40-345f-454c-8e23-77a427d9f850 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-f3231a40-345f-454c-8e23-77a427d9f850 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:52.840973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:52.842028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:52.842957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:52.842957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:52.842957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:52.842957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:52.842957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:52.842957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:52.842957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:52.842957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:52.843788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 532/1049] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:52.849984 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a594a5-d494-462c-85b8-3db6e3fcbe28 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:53.002377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a594a5-d494-462c-85b8-3db6e3fcbe28 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:53.045358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d8364-92ca-4ce2-8f22-346ab592e0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:53.046202 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d8364-92ca-4ce2-8f22-346ab592e0ef None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:53.046653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d8364-92ca-4ce2-8f22-346ab592e0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:53.047077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d8364-92ca-4ce2-8f22-346ab592e0ef None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:53.048128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d8364-92ca-4ce2-8f22-346ab592e0ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:53.048573 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d8364-92ca-4ce2-8f22-346ab592e0ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:53.050185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d8364-92ca-4ce2-8f22-346ab592e0ef None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:53.063597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 518/1050] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:53.069334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3231a40-345f-454c-8e23-77a427d9f850 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:53.151920 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:53.232140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3231a40-345f-454c-8e23-77a427d9f850 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:53.278736 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f616bc-60a2-4543-8a70-c38ff40746a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:53.279721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f616bc-60a2-4543-8a70-c38ff40746a0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:53.280153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f616bc-60a2-4543-8a70-c38ff40746a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:53.280582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f616bc-60a2-4543-8a70-c38ff40746a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:53.281949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f616bc-60a2-4543-8a70-c38ff40746a0 None admin] RBAC: Adding query filter params (role.id=ca62ca555176433484fd406e5442b7d2, scope.project.id=23546323ca33412fb388afe308a97499, user.id=c1966524091242e7a6cd2758027a0d4f) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:53.282427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f616bc-60a2-4543-8a70-c38ff40746a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:53.284770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f616bc-60a2-4543-8a70-c38ff40746a0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:53.302570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 533/1051] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:53 2022] GET /identity/v3/role_assignments?user.id=c1966524091242e7a6cd2758027a0d4f&scope.project.id=23546323ca33412fb388afe308a97499&role.id=ca62ca555176433484fd406e5442b7d2 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:54.820236 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06d4a9-6c79-4b76-a62d-7b53535d01b4 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:54.820771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06d4a9-6c79-4b76-a62d-7b53535d01b4 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:54.820855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06d4a9-6c79-4b76-a62d-7b53535d01b4 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:54.822351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 519/1052] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:54.827869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb2339-01ef-4bff-943e-3f70f5d758f8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:54.828477 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb2339-01ef-4bff-943e-3f70f5d758f8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:54.828477 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb2339-01ef-4bff-943e-3f70f5d758f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:54.847230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-abdb2339-01ef-4bff-943e-3f70f5d758f8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:54.848425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-abdb2339-01ef-4bff-943e-3f70f5d758f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:54.929871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 534/1053] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:54.956334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc815aa-4d2f-4b89-8122-b4f3f40c4363 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:54.956840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc815aa-4d2f-4b89-8122-b4f3f40c4363 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:54.957165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb2339-01ef-4bff-943e-3f70f5d758f8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:54.957310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc815aa-4d2f-4b89-8122-b4f3f40c4363 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:54.959113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 520/1054] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:54.964971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56491e-79cb-4a45-8e2a-f49ec9235730 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:54.965260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56491e-79cb-4a45-8e2a-f49ec9235730 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:54.965546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56491e-79cb-4a45-8e2a-f49ec9235730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:54.984277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-8b56491e-79cb-4a45-8e2a-f49ec9235730 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:54.985490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8b56491e-79cb-4a45-8e2a-f49ec9235730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:55.028096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:55.069234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:55.094607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 521/1055] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:55.107268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb2339-01ef-4bff-943e-3f70f5d758f8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:55.118797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56491e-79cb-4a45-8e2a-f49ec9235730 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:55.156299 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf41dd9-a139-4b5d-97db-43884da0bd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:55.156956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf41dd9-a139-4b5d-97db-43884da0bd51 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:55.157237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf41dd9-a139-4b5d-97db-43884da0bd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:55.157508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf41dd9-a139-4b5d-97db-43884da0bd51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:55.265826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56491e-79cb-4a45-8e2a-f49ec9235730 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:55.273141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf41dd9-a139-4b5d-97db-43884da0bd51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:55.275650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf41dd9-a139-4b5d-97db-43884da0bd51 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:55.322571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22fe29-3848-4f6b-8d48-4debd6924838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:55.323666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22fe29-3848-4f6b-8d48-4debd6924838 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:55.323941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22fe29-3848-4f6b-8d48-4debd6924838 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:55.324210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22fe29-3848-4f6b-8d48-4debd6924838 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:55.336533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22fe29-3848-4f6b-8d48-4debd6924838 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-2e22fe29-3848-4f6b-8d48-4debd6924838 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:55.345305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:55.346841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:55.347705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 522/1056] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:55.348437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:55.352415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22fe29-3848-4f6b-8d48-4debd6924838 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:55.411158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:55.421546 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 535/1057] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 15:13:55.427687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf41dd9-a139-4b5d-97db-43884da0bd51 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:55.491079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22fe29-3848-4f6b-8d48-4debd6924838 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:55.542255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c766ec-41f8-47cc-bfb5-805d7afb178b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:55.542893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c766ec-41f8-47cc-bfb5-805d7afb178b None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:55.543156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c766ec-41f8-47cc-bfb5-805d7afb178b None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:55.543434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c766ec-41f8-47cc-bfb5-805d7afb178b None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:55.544204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c766ec-41f8-47cc-bfb5-805d7afb178b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:55.544517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c766ec-41f8-47cc-bfb5-805d7afb178b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:55.546172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c766ec-41f8-47cc-bfb5-805d7afb178b None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:55.561153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 523/1058] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:13:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:55.569960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c766ec-41f8-47cc-bfb5-805d7afb178b None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:55.587749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf41dd9-a139-4b5d-97db-43884da0bd51 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:55.631188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f92eaf-f2e8-4e0e-ae31-7b4f56280ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:55.632179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f92eaf-f2e8-4e0e-ae31-7b4f56280ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:55.632597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f92eaf-f2e8-4e0e-ae31-7b4f56280ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:55.633001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f92eaf-f2e8-4e0e-ae31-7b4f56280ff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:55.646381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f92eaf-f2e8-4e0e-ae31-7b4f56280ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:55.648046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f92eaf-f2e8-4e0e-ae31-7b4f56280ff3 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:55.662171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c8f92eaf-f2e8-4e0e-ae31-7b4f56280ff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:55.675209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:55.676176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:55.677100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:55.677100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:55.677100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:55.677100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:55.681027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 536/1059] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:55.687982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f92eaf-f2e8-4e0e-ae31-7b4f56280ff3 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:55.699921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c766ec-41f8-47cc-bfb5-805d7afb178b None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:55.709038 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:55.743790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb832bf-dd7e-4e16-a33c-be0a74c64f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:55.744918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb832bf-dd7e-4e16-a33c-be0a74c64f14 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:55.745333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb832bf-dd7e-4e16-a33c-be0a74c64f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:55.745780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb832bf-dd7e-4e16-a33c-be0a74c64f14 None admin] PATH_INFO: `/v3/users/672de50bb93044359ecb99c72d2ca54b` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:55.760604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb832bf-dd7e-4e16-a33c-be0a74c64f14 None admin] RBAC: Authorizing `identity:get_user(user_id=672de50bb93044359ecb99c72d2ca54b)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:55.762060 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb832bf-dd7e-4e16-a33c-be0a74c64f14 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:55.763793 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 524/1060] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:13:55 2022] GET /identity/v3/users/672de50bb93044359ecb99c72d2ca54b => generated 285 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:55.765026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8361-0023-4636-afeb-519d208ba57c None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:55.765313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8361-0023-4636-afeb-519d208ba57c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:55.765591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8361-0023-4636-afeb-519d208ba57c None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:55.766844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 525/1061] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:55.769847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f8361-0023-4636-afeb-519d208ba57c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:55.805974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f92eaf-f2e8-4e0e-ae31-7b4f56280ff3 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:55.842192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f615654-a8fc-48bc-85be-e8a756cdee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:55.842963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f615654-a8fc-48bc-85be-e8a756cdee33 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:55.843256 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f615654-a8fc-48bc-85be-e8a756cdee33 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:55.843454 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f615654-a8fc-48bc-85be-e8a756cdee33 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:55.844127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f615654-a8fc-48bc-85be-e8a756cdee33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:55.844433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f615654-a8fc-48bc-85be-e8a756cdee33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:55.845796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f615654-a8fc-48bc-85be-e8a756cdee33 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:55.859877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 537/1062] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:55.861183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8db074-b429-4d37-af65-8a77ce87b8ba None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:55.861476 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8db074-b429-4d37-af65-8a77ce87b8ba None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:55.861751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8db074-b429-4d37-af65-8a77ce87b8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:55.882176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ac8db074-b429-4d37-af65-8a77ce87b8ba None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:55.883501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ac8db074-b429-4d37-af65-8a77ce87b8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:55.916985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f8361-0023-4636-afeb-519d208ba57c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:55.955281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:55.970338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 538/1063] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:13:55.971048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72407323-fc62-40f4-a148-05a27ff4ead3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:55.972151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72407323-fc62-40f4-a148-05a27ff4ead3 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:55.972699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72407323-fc62-40f4-a148-05a27ff4ead3 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:55.973265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72407323-fc62-40f4-a148-05a27ff4ead3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:55.998793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8db074-b429-4d37-af65-8a77ce87b8ba None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:56.019540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:56.070296 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72407323-fc62-40f4-a148-05a27ff4ead3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:56.070665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72407323-fc62-40f4-a148-05a27ff4ead3 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:56.186945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8db074-b429-4d37-af65-8a77ce87b8ba None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:56.193713 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 526/1064] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:13:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 15:13:56.204522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72407323-fc62-40f4-a148-05a27ff4ead3 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:56.230005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd56b3-c555-400d-8997-effde5d2bd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:56.230916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd56b3-c555-400d-8997-effde5d2bd60 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:56.231199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd56b3-c555-400d-8997-effde5d2bd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:56.231466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cd56b3-c555-400d-8997-effde5d2bd60 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:56.244046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cd56b3-c555-400d-8997-effde5d2bd60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-58cd56b3-c555-400d-8997-effde5d2bd60 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:56.252390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:56.253538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:56.254425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 539/1065] 10.222.0.93 () {56 vars in 1120 bytes} [Fri Jun 3 15:13:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:56.259932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd56b3-c555-400d-8997-effde5d2bd60 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:56.280188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:56.315261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:56.321827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72407323-fc62-40f4-a148-05a27ff4ead3 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:56.359291 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093db68-c025-4ad8-b4f9-03c41c8fb384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:56.360163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093db68-c025-4ad8-b4f9-03c41c8fb384 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:56.360482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093db68-c025-4ad8-b4f9-03c41c8fb384 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:56.360760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093db68-c025-4ad8-b4f9-03c41c8fb384 None admin] PATH_INFO: `/v3/projects/ad18ca11afd74e6daccb96c4ae260526` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:56.361983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093db68-c025-4ad8-b4f9-03c41c8fb384 None admin] RBAC: Authorizing `identity:get_project(project_id=ad18ca11afd74e6daccb96c4ae260526)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:56.363470 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093db68-c025-4ad8-b4f9-03c41c8fb384 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:56.365479 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 527/1066] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:13:56 2022] GET /identity/v3/projects/ad18ca11afd74e6daccb96c4ae260526 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:56.371099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cd56b3-c555-400d-8997-effde5d2bd60 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:56.371589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093db68-c025-4ad8-b4f9-03c41c8fb384 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:56.420987 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a9a9d-bfbf-4cf7-9f2d-74ce5960b99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:56.422032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a9a9d-bfbf-4cf7-9f2d-74ce5960b99f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:56.422350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a9a9d-bfbf-4cf7-9f2d-74ce5960b99f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:56.422667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a9a9d-bfbf-4cf7-9f2d-74ce5960b99f None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:56.423436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a9a9d-bfbf-4cf7-9f2d-74ce5960b99f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:56.423750 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a9a9d-bfbf-4cf7-9f2d-74ce5960b99f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:56.425146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a9a9d-bfbf-4cf7-9f2d-74ce5960b99f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:56.433786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 540/1067] 10.222.0.93 () {56 vars in 1115 bytes} [Fri Jun 3 15:13:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:56.440513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a9a9d-bfbf-4cf7-9f2d-74ce5960b99f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:56.486234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093db68-c025-4ad8-b4f9-03c41c8fb384 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:56.521927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14552c-160a-4b06-a526-1c476f3a2200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:56.522593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14552c-160a-4b06-a526-1c476f3a2200 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:56.522890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14552c-160a-4b06-a526-1c476f3a2200 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:56.523163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14552c-160a-4b06-a526-1c476f3a2200 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:56.524639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14552c-160a-4b06-a526-1c476f3a2200 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=ad18ca11afd74e6daccb96c4ae260526, user.id=672de50bb93044359ecb99c72d2ca54b) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:56.524969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14552c-160a-4b06-a526-1c476f3a2200 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:56.526511 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14552c-160a-4b06-a526-1c476f3a2200 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:56.528393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:56.537540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 528/1068] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:13:56 2022] GET /identity/v3/role_assignments?user.id=672de50bb93044359ecb99c72d2ca54b&scope.project.id=ad18ca11afd74e6daccb96c4ae260526&role.id=d67707782358417ba9f25b2793ca2890 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:56.575107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a9a9d-bfbf-4cf7-9f2d-74ce5960b99f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:56.595602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:56.630653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fbcfb-f71e-4363-b8ed-321cef89ba2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:56.631470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fbcfb-f71e-4363-b8ed-321cef89ba2a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:56.631752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fbcfb-f71e-4363-b8ed-321cef89ba2a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:56.632028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fbcfb-f71e-4363-b8ed-321cef89ba2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:56.645170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fbcfb-f71e-4363-b8ed-321cef89ba2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:56.646710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fbcfb-f71e-4363-b8ed-321cef89ba2a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-3f3fbcfb-f71e-4363-b8ed-321cef89ba2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:56.666988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:56.675414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:56.677537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:56.677537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:56.677537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:56.677537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:56.677537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 541/1069] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:13:56.683164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14552c-160a-4b06-a526-1c476f3a2200 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:56.795166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14552c-160a-4b06-a526-1c476f3a2200 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:56.833363 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e9372-3ead-4bf5-b4e3-03cd301abfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:56.833914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e9372-3ead-4bf5-b4e3-03cd301abfe8 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:56.834196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e9372-3ead-4bf5-b4e3-03cd301abfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:56.834749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81e9372-3ead-4bf5-b4e3-03cd301abfe8 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:56.836039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e9372-3ead-4bf5-b4e3-03cd301abfe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:56.837038 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e9372-3ead-4bf5-b4e3-03cd301abfe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:56.839396 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81e9372-3ead-4bf5-b4e3-03cd301abfe8 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:56.852979 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 529/1070] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:56.858852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fbcfb-f71e-4363-b8ed-321cef89ba2a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:57.011468 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fbcfb-f71e-4363-b8ed-321cef89ba2a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:57.051237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9f292-0f9b-45a0-841d-3b66a5b983ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:57.052037 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9f292-0f9b-45a0-841d-3b66a5b983ee None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:57.052310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9f292-0f9b-45a0-841d-3b66a5b983ee None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:57.052595 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9f292-0f9b-45a0-841d-3b66a5b983ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:57.060747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9f292-0f9b-45a0-841d-3b66a5b983ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:57.062203 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9f292-0f9b-45a0-841d-3b66a5b983ee None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c2a9f292-0f9b-45a0-841d-3b66a5b983ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:13:57.078525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:57.079823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:57.081747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:57.081747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:57.081747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:13:57.081747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:13:57.081747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:13:57.081747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:13:57.081747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:13:57.081747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:13:57.081747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:13:57.081747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:57.081747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 542/1071] 10.222.0.93 () {56 vars in 1093 bytes} [Fri Jun 3 15:13:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:57.087128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e9372-3ead-4bf5-b4e3-03cd301abfe8 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:57.161696 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:57.206597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81e9372-3ead-4bf5-b4e3-03cd301abfe8 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:57.242728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62ee2e-1eaf-4fe5-bc3f-a20c2c2356d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:57.243575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62ee2e-1eaf-4fe5-bc3f-a20c2c2356d5 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:57.243903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62ee2e-1eaf-4fe5-bc3f-a20c2c2356d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:57.244235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62ee2e-1eaf-4fe5-bc3f-a20c2c2356d5 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:57.245122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62ee2e-1eaf-4fe5-bc3f-a20c2c2356d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:57.245498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62ee2e-1eaf-4fe5-bc3f-a20c2c2356d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:57.247047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62ee2e-1eaf-4fe5-bc3f-a20c2c2356d5 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:57.266698 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 530/1072] 10.222.0.93 () {56 vars in 1133 bytes} [Fri Jun 3 15:13:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:57.273361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9f292-0f9b-45a0-841d-3b66a5b983ee None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:57.307905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:57.396621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9f292-0f9b-45a0-841d-3b66a5b983ee None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:57.438708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22896439-20de-4e6c-addf-de0b98858d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:57.439495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22896439-20de-4e6c-addf-de0b98858d2b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:57.439777 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22896439-20de-4e6c-addf-de0b98858d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:57.440055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22896439-20de-4e6c-addf-de0b98858d2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:57.452563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22896439-20de-4e6c-addf-de0b98858d2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:57.454023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22896439-20de-4e6c-addf-de0b98858d2b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-22896439-20de-4e6c-addf-de0b98858d2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:13:57.475417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:57.476457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:57.477473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:13:57.477473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:13:57.477473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:13:57.477473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:57.477893 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 543/1073] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:57.483505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62ee2e-1eaf-4fe5-bc3f-a20c2c2356d5 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:57.590578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62ee2e-1eaf-4fe5-bc3f-a20c2c2356d5 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:57.609012 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:57.629628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5233a-439f-40dc-9a78-660f252add56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:57.630288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5233a-439f-40dc-9a78-660f252add56 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:57.630620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5233a-439f-40dc-9a78-660f252add56 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:57.631093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5233a-439f-40dc-9a78-660f252add56 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:57.631816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5233a-439f-40dc-9a78-660f252add56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:57.632119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5233a-439f-40dc-9a78-660f252add56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:57.634680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5233a-439f-40dc-9a78-660f252add56 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:57.647748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 531/1074] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:57.655600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22896439-20de-4e6c-addf-de0b98858d2b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:57.766040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:57.791172 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22896439-20de-4e6c-addf-de0b98858d2b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:57.828599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ab907-3637-4a15-9b82-a0a52e682da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:57.829611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ab907-3637-4a15-9b82-a0a52e682da7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:57.829973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ab907-3637-4a15-9b82-a0a52e682da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:57.830294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ab907-3637-4a15-9b82-a0a52e682da7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:57.839930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ab907-3637-4a15-9b82-a0a52e682da7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:57.841393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ab907-3637-4a15-9b82-a0a52e682da7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-000ab907-3637-4a15-9b82-a0a52e682da7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:13:57.856508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:57.857855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:57.859911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:13:57.859911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:13:57.859911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:13:57.859911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:13:57.859911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:13:57.859911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:13:57.859911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:13:57.859911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:13:57.859911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 544/1075] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:13:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:57.864236 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5233a-439f-40dc-9a78-660f252add56 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:57.971871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5233a-439f-40dc-9a78-660f252add56 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:58.007733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907ddfb-2de8-4661-89fa-db6bd34802ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:58.008956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907ddfb-2de8-4661-89fa-db6bd34802ce None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:58.009300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907ddfb-2de8-4661-89fa-db6bd34802ce None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:58.009632 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907ddfb-2de8-4661-89fa-db6bd34802ce None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:58.010622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907ddfb-2de8-4661-89fa-db6bd34802ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:58.011074 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907ddfb-2de8-4661-89fa-db6bd34802ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:58.014286 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907ddfb-2de8-4661-89fa-db6bd34802ce None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:58.032695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 532/1076] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:13:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:58.040396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ab907-3637-4a15-9b82-a0a52e682da7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:58.207274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ab907-3637-4a15-9b82-a0a52e682da7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:58.256683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b18f5d-a06a-46fb-a66c-ce9d3cf38a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:58.257567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b18f5d-a06a-46fb-a66c-ce9d3cf38a37 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:58.257850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b18f5d-a06a-46fb-a66c-ce9d3cf38a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:58.258645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b18f5d-a06a-46fb-a66c-ce9d3cf38a37 None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/c1966524091242e7a6cd2758027a0d4f/roles/ca62ca555176433484fd406e5442b7d2` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:58.260897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b18f5d-a06a-46fb-a66c-ce9d3cf38a37 None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=c1966524091242e7a6cd2758027a0d4f, role_id=ca62ca555176433484fd406e5442b7d2)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:58.262363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b18f5d-a06a-46fb-a66c-ce9d3cf38a37 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:58.265609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:58.283914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 545/1077] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:13:58 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/c1966524091242e7a6cd2758027a0d4f/roles/ca62ca555176433484fd406e5442b7d2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:13:58.916876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5d803-a0c7-484a-905c-43d782f50d23 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:58.917717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5d803-a0c7-484a-905c-43d782f50d23 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:58.917717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5d803-a0c7-484a-905c-43d782f50d23 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:58.919015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 533/1078] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:13:58.924150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac4c5c-d5fa-49de-8406-d9ded876ec40 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:58.924617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac4c5c-d5fa-49de-8406-d9ded876ec40 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:58.924930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac4c5c-d5fa-49de-8406-d9ded876ec40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:58.945571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-afac4c5c-d5fa-49de-8406-d9ded876ec40 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:13:58.946736 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-afac4c5c-d5fa-49de-8406-d9ded876ec40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:13:59.016943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 546/1079] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:13:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:13:59.040326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5d803-a0c7-484a-905c-43d782f50d23 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:59.078275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:59.148199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5d803-a0c7-484a-905c-43d782f50d23 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:59.184404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623b450-9554-4124-85f5-1d1a0a8febe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:59.185368 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623b450-9554-4124-85f5-1d1a0a8febe9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:59.185647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623b450-9554-4124-85f5-1d1a0a8febe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:59.185923 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623b450-9554-4124-85f5-1d1a0a8febe9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:59.196099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623b450-9554-4124-85f5-1d1a0a8febe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-b623b450-9554-4124-85f5-1d1a0a8febe9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:13:59.203524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:13:59.204614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:13:59.205717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 534/1080] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:13:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:13:59.211028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac4c5c-d5fa-49de-8406-d9ded876ec40 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:59.308909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac4c5c-d5fa-49de-8406-d9ded876ec40 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:59.335081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:59.349079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ef906-6bd8-4e73-8c83-75137d85e1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:59.349805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ef906-6bd8-4e73-8c83-75137d85e1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:59.350080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ef906-6bd8-4e73-8c83-75137d85e1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:59.350349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ef906-6bd8-4e73-8c83-75137d85e1b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:59.351125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ef906-6bd8-4e73-8c83-75137d85e1b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:13:59.351425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ef906-6bd8-4e73-8c83-75137d85e1b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:59.352819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ef906-6bd8-4e73-8c83-75137d85e1b6 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:59.363718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 547/1081] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:59.370312 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623b450-9554-4124-85f5-1d1a0a8febe9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:59.467970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:13:59.474433 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623b450-9554-4124-85f5-1d1a0a8febe9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:59.508774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd1d98-4818-4566-ad9d-b88c02b5dca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:59.509645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd1d98-4818-4566-ad9d-b88c02b5dca5 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:59.509926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd1d98-4818-4566-ad9d-b88c02b5dca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:59.510205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd1d98-4818-4566-ad9d-b88c02b5dca5 None admin] PATH_INFO: `/v3/users/672de50bb93044359ecb99c72d2ca54b` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:59.510205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd1d98-4818-4566-ad9d-b88c02b5dca5 None admin] RBAC: Authorizing `identity:get_user(user_id=672de50bb93044359ecb99c72d2ca54b)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:59.510205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd1d98-4818-4566-ad9d-b88c02b5dca5 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:59.510205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 535/1082] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:13:59 2022] GET /identity/v3/users/672de50bb93044359ecb99c72d2ca54b => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:13:59.520368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ef906-6bd8-4e73-8c83-75137d85e1b6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:59.625303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ef906-6bd8-4e73-8c83-75137d85e1b6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:59.660733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa660cb3-f189-4ba8-acbd-f7a7299a30d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:59.661845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa660cb3-f189-4ba8-acbd-f7a7299a30d0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:59.662131 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa660cb3-f189-4ba8-acbd-f7a7299a30d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:59.662411 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa660cb3-f189-4ba8-acbd-f7a7299a30d0 None admin] PATH_INFO: `/v3/projects/ad18ca11afd74e6daccb96c4ae260526` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:59.663629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa660cb3-f189-4ba8-acbd-f7a7299a30d0 None admin] RBAC: Authorizing `identity:get_project(project_id=ad18ca11afd74e6daccb96c4ae260526)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:59.665594 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa660cb3-f189-4ba8-acbd-f7a7299a30d0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:59.667319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 548/1083] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:13:59 2022] GET /identity/v3/projects/ad18ca11afd74e6daccb96c4ae260526 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:13:59.673304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd1d98-4818-4566-ad9d-b88c02b5dca5 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:13:59.776329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd1d98-4818-4566-ad9d-b88c02b5dca5 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:13:59.811054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9971f8e-0644-4682-abb6-f25a40e7ba40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:13:59.811938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9971f8e-0644-4682-abb6-f25a40e7ba40 None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:13:59.812221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9971f8e-0644-4682-abb6-f25a40e7ba40 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:13:59.812497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9971f8e-0644-4682-abb6-f25a40e7ba40 None admin] PATH_INFO: `/v3/projects/ad18ca11afd74e6daccb96c4ae260526/users/672de50bb93044359ecb99c72d2ca54b/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:13:59.814437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9971f8e-0644-4682-abb6-f25a40e7ba40 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad18ca11afd74e6daccb96c4ae260526, user_id=672de50bb93044359ecb99c72d2ca54b, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:13:59.815918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9971f8e-0644-4682-abb6-f25a40e7ba40 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:13:59.828973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 536/1084] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:13:59 2022] PUT /identity/v3/projects/ad18ca11afd74e6daccb96c4ae260526/users/672de50bb93044359ecb99c72d2ca54b/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:14:00.596088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea44479-4f09-476e-8b86-08396573d673 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:00.596719 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea44479-4f09-476e-8b86-08396573d673 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:00.596719 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea44479-4f09-476e-8b86-08396573d673 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:00.598181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 549/1085] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:00.604121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a43054-68ab-488e-a3ca-013e2b99f09f None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:00.604541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a43054-68ab-488e-a3ca-013e2b99f09f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:00.604684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a43054-68ab-488e-a3ca-013e2b99f09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:00.622250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c7a43054-68ab-488e-a3ca-013e2b99f09f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:00.623466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c7a43054-68ab-488e-a3ca-013e2b99f09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:00.642050 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:00.701709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 537/1086] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:00.727288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea44479-4f09-476e-8b86-08396573d673 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:00.804191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:00.853308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea44479-4f09-476e-8b86-08396573d673 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:00.887495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22382f6-31dc-49ff-9f66-a963b07e4a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:00.888321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22382f6-31dc-49ff-9f66-a963b07e4a0d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:00.888574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22382f6-31dc-49ff-9f66-a963b07e4a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:00.888843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22382f6-31dc-49ff-9f66-a963b07e4a0d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:00.898957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22382f6-31dc-49ff-9f66-a963b07e4a0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-e22382f6-31dc-49ff-9f66-a963b07e4a0d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:14:00.907091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:00.909070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:14:00.910806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 550/1087] 10.222.0.93 () {56 vars in 1120 bytes} [Fri Jun 3 15:14:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:14:00.914624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a43054-68ab-488e-a3ca-013e2b99f09f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:01.015181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a43054-68ab-488e-a3ca-013e2b99f09f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:01.054362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357d9d2-ddce-40b4-88fd-765aece19127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:01.055639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d9d2-ddce-40b4-88fd-765aece19127 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:01.055916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d9d2-ddce-40b4-88fd-765aece19127 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:01.056195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357d9d2-ddce-40b4-88fd-765aece19127 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:01.056939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357d9d2-ddce-40b4-88fd-765aece19127 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:01.057244 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357d9d2-ddce-40b4-88fd-765aece19127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:01.058654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357d9d2-ddce-40b4-88fd-765aece19127 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:01.066968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 538/1088] 10.222.0.93 () {56 vars in 1115 bytes} [Fri Jun 3 15:14:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:14:01.072770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22382f6-31dc-49ff-9f66-a963b07e4a0d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:01.189935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22382f6-31dc-49ff-9f66-a963b07e4a0d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:01.205591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:01.227998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f1d37-db02-4977-9b92-d35a4048a18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:01.229319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f1d37-db02-4977-9b92-d35a4048a18f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:01.229809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f1d37-db02-4977-9b92-d35a4048a18f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:01.230289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f1d37-db02-4977-9b92-d35a4048a18f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:01.248363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f1d37-db02-4977-9b92-d35a4048a18f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:01.251061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f1d37-db02-4977-9b92-d35a4048a18f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-3d4f1d37-db02-4977-9b92-d35a4048a18f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:14:01.272917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:01.274081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:01.275282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:14:01.275282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:14:01.275282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:14:01.275282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:14:01.278666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 551/1089] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:14:01.279982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357d9d2-ddce-40b4-88fd-765aece19127 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:01.298786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:01.379320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357d9d2-ddce-40b4-88fd-765aece19127 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:01.412583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260966ee-a5f6-446b-87f7-520b315132a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:01.413240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260966ee-a5f6-446b-87f7-520b315132a6 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:01.413521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260966ee-a5f6-446b-87f7-520b315132a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:01.413795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260966ee-a5f6-446b-87f7-520b315132a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:01.415098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260966ee-a5f6-446b-87f7-520b315132a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:01.415402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260966ee-a5f6-446b-87f7-520b315132a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:01.416769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260966ee-a5f6-446b-87f7-520b315132a6 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:01.428371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 539/1090] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:01.434073 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f1d37-db02-4977-9b92-d35a4048a18f None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:01.532102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f1d37-db02-4977-9b92-d35a4048a18f None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:01.565016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb01ab-c5b1-4428-a8b0-6b52a6f54e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:01.565803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb01ab-c5b1-4428-a8b0-6b52a6f54e5b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:01.566086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb01ab-c5b1-4428-a8b0-6b52a6f54e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:01.566361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb01ab-c5b1-4428-a8b0-6b52a6f54e5b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:01.580036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb01ab-c5b1-4428-a8b0-6b52a6f54e5b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:01.581487 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb01ab-c5b1-4428-a8b0-6b52a6f54e5b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-01eb01ab-c5b1-4428-a8b0-6b52a6f54e5b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:14:01.595927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:01.597382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:01.598347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:01.598347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:01.598347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:14:01.598347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:14:01.598347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:14:01.598347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:14:01.598347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:14:01.598347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:14:01.598347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 03 15:14:01.598347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:14:01.598347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 552/1091] 10.222.0.93 () {56 vars in 1093 bytes} [Fri Jun 3 15:14:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:14:01.603665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260966ee-a5f6-446b-87f7-520b315132a6 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:01.684437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:01.707613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260966ee-a5f6-446b-87f7-520b315132a6 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:01.742198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b39a06-d1e3-42dc-a3eb-1aab01855d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:01.743031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39a06-d1e3-42dc-a3eb-1aab01855d62 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:01.743186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39a06-d1e3-42dc-a3eb-1aab01855d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:01.743457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b39a06-d1e3-42dc-a3eb-1aab01855d62 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:01.744290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b39a06-d1e3-42dc-a3eb-1aab01855d62 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:01.744598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b39a06-d1e3-42dc-a3eb-1aab01855d62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:01.746031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b39a06-d1e3-42dc-a3eb-1aab01855d62 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:01.763684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 540/1092] 10.222.0.93 () {56 vars in 1133 bytes} [Fri Jun 3 15:14:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:01.770276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb01ab-c5b1-4428-a8b0-6b52a6f54e5b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:01.821661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:01.896826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb01ab-c5b1-4428-a8b0-6b52a6f54e5b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:01.931331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4278bfba-2039-45e4-a7e4-124904105b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:01.931979 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278bfba-2039-45e4-a7e4-124904105b4b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:01.932264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278bfba-2039-45e4-a7e4-124904105b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:01.932540 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4278bfba-2039-45e4-a7e4-124904105b4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.002168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278bfba-2039-45e4-a7e4-124904105b4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:02.003668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4278bfba-2039-45e4-a7e4-124904105b4b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:02.031453 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:02.104151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 553/1093] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:14:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:14:02.109954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b39a06-d1e3-42dc-a3eb-1aab01855d62 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:02.130909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc028c-65b3-41ca-a97a-f0c6535904a5 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:02.131204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc028c-65b3-41ca-a97a-f0c6535904a5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:02.131490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc028c-65b3-41ca-a97a-f0c6535904a5 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.132883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 554/1094] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:14:02.138437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ac2fd-a173-4b28-957f-fe81d702de31 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:02.138871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ac2fd-a173-4b28-957f-fe81d702de31 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:02.139046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ac2fd-a173-4b28-957f-fe81d702de31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.155112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d00ac2fd-a173-4b28-957f-fe81d702de31 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:02.156300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d00ac2fd-a173-4b28-957f-fe81d702de31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:02.226761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b39a06-d1e3-42dc-a3eb-1aab01855d62 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:02.232478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 555/1095] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:14:02.257539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ac2fd-a173-4b28-957f-fe81d702de31 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:02.263305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa47939-b003-4c79-a979-7aff6a7ea1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:02.264101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa47939-b003-4c79-a979-7aff6a7ea1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:02.264382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa47939-b003-4c79-a979-7aff6a7ea1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:02.264654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa47939-b003-4c79-a979-7aff6a7ea1d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.266602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:02.281657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa47939-b003-4c79-a979-7aff6a7ea1d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:02.284199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa47939-b003-4c79-a979-7aff6a7ea1d2 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-7fa47939-b003-4c79-a979-7aff6a7ea1d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:14:02.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:02.309122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:02.312840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:14:02.312840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:14:02.312840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:14:02.312840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:14:02.312840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 541/1096] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:14:02.312840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa47939-b003-4c79-a979-7aff6a7ea1d2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:02.327833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:02.370237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ac2fd-a173-4b28-957f-fe81d702de31 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:02.404130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75ad7d-024c-422e-ad45-e49f401df267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:02.404918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75ad7d-024c-422e-ad45-e49f401df267 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:02.405233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75ad7d-024c-422e-ad45-e49f401df267 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:02.405513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd75ad7d-024c-422e-ad45-e49f401df267 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.411275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa47939-b003-4c79-a979-7aff6a7ea1d2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:02.415779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd75ad7d-024c-422e-ad45-e49f401df267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-fd75ad7d-024c-422e-ad45-e49f401df267 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:14:02.423580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:02.424656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:14:02.425539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 556/1097] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:14:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:14:02.431165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75ad7d-024c-422e-ad45-e49f401df267 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:02.444964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fd3cf-be5d-42bf-bee7-7f8dd19f8242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:02.446182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fd3cf-be5d-42bf-bee7-7f8dd19f8242 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:02.446460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fd3cf-be5d-42bf-bee7-7f8dd19f8242 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:02.446761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fd3cf-be5d-42bf-bee7-7f8dd19f8242 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.447483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fd3cf-be5d-42bf-bee7-7f8dd19f8242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:02.447786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fd3cf-be5d-42bf-bee7-7f8dd19f8242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:02.449164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fd3cf-be5d-42bf-bee7-7f8dd19f8242 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:02.460314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 542/1098] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:02.465888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fd3cf-be5d-42bf-bee7-7f8dd19f8242 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:02.518470 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:02.538103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd75ad7d-024c-422e-ad45-e49f401df267 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:02.555792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:02.573915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c09f5-c879-421a-8939-c2b169b640ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:02.574608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c09f5-c879-421a-8939-c2b169b640ba None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:02.574897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c09f5-c879-421a-8939-c2b169b640ba None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:02.575175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c09f5-c879-421a-8939-c2b169b640ba None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.575470 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fd3cf-be5d-42bf-bee7-7f8dd19f8242 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:02.575923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c09f5-c879-421a-8939-c2b169b640ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:02.576231 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c09f5-c879-421a-8939-c2b169b640ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:02.577620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c09f5-c879-421a-8939-c2b169b640ba None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:02.585976 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 557/1099] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:14:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:14:02.591650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c09f5-c879-421a-8939-c2b169b640ba None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:02.609606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a012-c1b4-469a-913d-5e51faa1cf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:02.610389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a012-c1b4-469a-913d-5e51faa1cf32 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:02.610715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a012-c1b4-469a-913d-5e51faa1cf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:02.610981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102a012-c1b4-469a-913d-5e51faa1cf32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.618273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102a012-c1b4-469a-913d-5e51faa1cf32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:02.619749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102a012-c1b4-469a-913d-5e51faa1cf32 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-f102a012-c1b4-469a-913d-5e51faa1cf32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:14:02.634249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:02.635387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:02.636519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:14:02.636519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:14:02.636519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:14:02.636519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:14:02.636519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:14:02.636519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:14:02.636519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:14:02.636519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:14:02.636519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 543/1100] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:14:02.641021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a012-c1b4-469a-913d-5e51faa1cf32 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:02.697114 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c09f5-c879-421a-8939-c2b169b640ba None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:02.730712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117a94-9eaf-46ba-ab2b-23b57f7225cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:02.731592 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117a94-9eaf-46ba-ab2b-23b57f7225cf None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:02.731878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117a94-9eaf-46ba-ab2b-23b57f7225cf None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:02.732153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117a94-9eaf-46ba-ab2b-23b57f7225cf None admin] PATH_INFO: `/v3/users/672de50bb93044359ecb99c72d2ca54b` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.733362 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70117a94-9eaf-46ba-ab2b-23b57f7225cf None admin] RBAC: Authorizing `identity:get_user(user_id=672de50bb93044359ecb99c72d2ca54b)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:02.734831 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70117a94-9eaf-46ba-ab2b-23b57f7225cf None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:02.736468 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 558/1101] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:02 2022] GET /identity/v3/users/672de50bb93044359ecb99c72d2ca54b => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:14:02.743215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117a94-9eaf-46ba-ab2b-23b57f7225cf None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:02.745536 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102a012-c1b4-469a-913d-5e51faa1cf32 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:02.753558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:02.780917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392e7e8-9478-4d75-8251-c375735a426a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:02.781579 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392e7e8-9478-4d75-8251-c375735a426a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:02.781869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392e7e8-9478-4d75-8251-c375735a426a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:02.782143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392e7e8-9478-4d75-8251-c375735a426a None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.782969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392e7e8-9478-4d75-8251-c375735a426a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:02.783281 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392e7e8-9478-4d75-8251-c375735a426a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:02.784711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a392e7e8-9478-4d75-8251-c375735a426a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:02.791999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:02.797697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 544/1102] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:14:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:14:02.803928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392e7e8-9478-4d75-8251-c375735a426a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:02.845328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117a94-9eaf-46ba-ab2b-23b57f7225cf None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:02.880608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba145143-b458-4dda-8bad-9875aa5d4c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:02.881517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba145143-b458-4dda-8bad-9875aa5d4c01 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:02.881805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba145143-b458-4dda-8bad-9875aa5d4c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:02.882084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba145143-b458-4dda-8bad-9875aa5d4c01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.903959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a392e7e8-9478-4d75-8251-c375735a426a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:02.938619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22410828-d23e-4117-810e-9c9ed5c37018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:02.939270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22410828-d23e-4117-810e-9c9ed5c37018 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:02.939551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22410828-d23e-4117-810e-9c9ed5c37018 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:02.939823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22410828-d23e-4117-810e-9c9ed5c37018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:02.940648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22410828-d23e-4117-810e-9c9ed5c37018 None admin] RBAC: Adding query filter params (role.id=ca62ca555176433484fd406e5442b7d2, scope.project.id=23546323ca33412fb388afe308a97499, user.id=c1966524091242e7a6cd2758027a0d4f) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:02.940954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22410828-d23e-4117-810e-9c9ed5c37018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:02.942369 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22410828-d23e-4117-810e-9c9ed5c37018 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:02.948241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 545/1103] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:14:02 2022] GET /identity/v3/role_assignments?user.id=c1966524091242e7a6cd2758027a0d4f&scope.project.id=23546323ca33412fb388afe308a97499&role.id=ca62ca555176433484fd406e5442b7d2 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:14:02.951852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba145143-b458-4dda-8bad-9875aa5d4c01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:02.953334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba145143-b458-4dda-8bad-9875aa5d4c01 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:03.005160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:03.061001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 559/1104] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:14:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 15:14:03.067091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22410828-d23e-4117-810e-9c9ed5c37018 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:03.102614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:03.183261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22410828-d23e-4117-810e-9c9ed5c37018 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:03.234382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2cfa8-4d0b-43c1-9a02-a29b5ce45332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:03.235905 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2cfa8-4d0b-43c1-9a02-a29b5ce45332 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:03.236396 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2cfa8-4d0b-43c1-9a02-a29b5ce45332 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:03.236876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2cfa8-4d0b-43c1-9a02-a29b5ce45332 None admin] PATH_INFO: `/v3/projects/ad18ca11afd74e6daccb96c4ae260526` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:03.238764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2cfa8-4d0b-43c1-9a02-a29b5ce45332 None admin] RBAC: Authorizing `identity:get_project(project_id=ad18ca11afd74e6daccb96c4ae260526)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:03.240518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2cfa8-4d0b-43c1-9a02-a29b5ce45332 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:03.248772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 546/1105] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:14:03 2022] GET /identity/v3/projects/ad18ca11afd74e6daccb96c4ae260526 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:03.250851 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba145143-b458-4dda-8bad-9875aa5d4c01 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:03.381001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba145143-b458-4dda-8bad-9875aa5d4c01 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:03.426834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27d215-cb74-493d-84af-2a8c50a3f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:03.427497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27d215-cb74-493d-84af-2a8c50a3f896 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:03.427776 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27d215-cb74-493d-84af-2a8c50a3f896 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:03.428053 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27d215-cb74-493d-84af-2a8c50a3f896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:03.428886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27d215-cb74-493d-84af-2a8c50a3f896 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=ad18ca11afd74e6daccb96c4ae260526, user.id=672de50bb93044359ecb99c72d2ca54b) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:03.429196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27d215-cb74-493d-84af-2a8c50a3f896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:03.430686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27d215-cb74-493d-84af-2a8c50a3f896 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:03.441695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 560/1106] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:14:03 2022] GET /identity/v3/role_assignments?user.id=672de50bb93044359ecb99c72d2ca54b&scope.project.id=ad18ca11afd74e6daccb96c4ae260526&role.id=d67707782358417ba9f25b2793ca2890 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:05.800576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb726d9-23f5-4846-9c0d-935c3449618d None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:05.801298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb726d9-23f5-4846-9c0d-935c3449618d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:05.801298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb726d9-23f5-4846-9c0d-935c3449618d None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:05.802556 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 547/1107] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:05.809050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641d72d-5837-4ac5-a3d0-d2469ba7205c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:05.811674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641d72d-5837-4ac5-a3d0-d2469ba7205c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:05.811674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641d72d-5837-4ac5-a3d0-d2469ba7205c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:05.826591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b641d72d-5837-4ac5-a3d0-d2469ba7205c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:05.827807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b641d72d-5837-4ac5-a3d0-d2469ba7205c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:05.835604 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:05.926135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 561/1108] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:05.951589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb726d9-23f5-4846-9c0d-935c3449618d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:06.063544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:06.068712 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb726d9-23f5-4846-9c0d-935c3449618d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:06.107245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b455b7-d9c3-440c-bdf4-2f0153ca4308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:06.108028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b455b7-d9c3-440c-bdf4-2f0153ca4308 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:06.108309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b455b7-d9c3-440c-bdf4-2f0153ca4308 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:06.108578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b455b7-d9c3-440c-bdf4-2f0153ca4308 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:06.109738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b455b7-d9c3-440c-bdf4-2f0153ca4308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:06.111196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b455b7-d9c3-440c-bdf4-2f0153ca4308 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:06.112821 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 548/1109] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:14:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:14:06.118778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641d72d-5837-4ac5-a3d0-d2469ba7205c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:06.228970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641d72d-5837-4ac5-a3d0-d2469ba7205c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:06.285276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3623f3-2a53-4985-ad5e-f5d728728590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:06.286249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3623f3-2a53-4985-ad5e-f5d728728590 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:06.286561 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3623f3-2a53-4985-ad5e-f5d728728590 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:06.286837 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3623f3-2a53-4985-ad5e-f5d728728590 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:06.287610 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3623f3-2a53-4985-ad5e-f5d728728590 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:06.289183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3623f3-2a53-4985-ad5e-f5d728728590 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:06.305350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:06.313556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 562/1110] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:14:06 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:14:08.686076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734daad-5be3-4379-b174-3c754440ed7a None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:08.686600 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734daad-5be3-4379-b174-3c754440ed7a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:08.686734 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734daad-5be3-4379-b174-3c754440ed7a None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:08.688128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 549/1111] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:08.694223 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d981e49-e26c-47d0-9ec7-2a5d0027fe11 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:08.695370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d981e49-e26c-47d0-9ec7-2a5d0027fe11 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:08.696865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d981e49-e26c-47d0-9ec7-2a5d0027fe11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:08.714558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3d981e49-e26c-47d0-9ec7-2a5d0027fe11 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:08.715744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3d981e49-e26c-47d0-9ec7-2a5d0027fe11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:08.784533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 563/1112] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:14:08.810225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734daad-5be3-4379-b174-3c754440ed7a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:08.969057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734daad-5be3-4379-b174-3c754440ed7a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:09.005124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2625a-a44e-4dee-ac8f-ae99bd5452a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:09.005972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2625a-a44e-4dee-ac8f-ae99bd5452a9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:09.006254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2625a-a44e-4dee-ac8f-ae99bd5452a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:09.006563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2625a-a44e-4dee-ac8f-ae99bd5452a9 None admin] PATH_INFO: `/v3/roles/f4461b279e86422b83e29521e7ca29d0` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:09.008075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2625a-a44e-4dee-ac8f-ae99bd5452a9 None admin] RBAC: Authorizing `identity:get_role(role_id=f4461b279e86422b83e29521e7ca29d0)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:09.009449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2625a-a44e-4dee-ac8f-ae99bd5452a9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:09.010915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 550/1113] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:08 2022] GET /identity/v3/roles/f4461b279e86422b83e29521e7ca29d0 => generated 223 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:09.019508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d981e49-e26c-47d0-9ec7-2a5d0027fe11 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:09.133274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d981e49-e26c-47d0-9ec7-2a5d0027fe11 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:09.160176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:09.169227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9ddb2-8073-4ca8-903a-499e7560f0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:09.170184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9ddb2-8073-4ca8-903a-499e7560f0ae None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:09.170467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9ddb2-8073-4ca8-903a-499e7560f0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:09.170863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9ddb2-8073-4ca8-903a-499e7560f0ae None admin] PATH_INFO: `/v3/users/d98136f09cce4289be4ef67b3efc34b5` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:09.184580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9ddb2-8073-4ca8-903a-499e7560f0ae None admin] RBAC: Authorizing `identity:get_user(user_id=d98136f09cce4289be4ef67b3efc34b5)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:09.186098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9ddb2-8073-4ca8-903a-499e7560f0ae None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:09.187796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 564/1114] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:09 2022] GET /identity/v3/users/d98136f09cce4289be4ef67b3efc34b5 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:09.196644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2625a-a44e-4dee-ac8f-ae99bd5452a9 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:09.236232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:09.310161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2625a-a44e-4dee-ac8f-ae99bd5452a9 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:09.345336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d2eba-552f-4665-b466-c54d5f942788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:09.345981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d2eba-552f-4665-b466-c54d5f942788 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:09.346269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d2eba-552f-4665-b466-c54d5f942788 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:09.346598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d2eba-552f-4665-b466-c54d5f942788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:09.418809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d2eba-552f-4665-b466-c54d5f942788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:09.420265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d2eba-552f-4665-b466-c54d5f942788 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:09.455088 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:09.529013 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 551/1115] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:14:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:14:09.535760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9ddb2-8073-4ca8-903a-499e7560f0ae None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:09.634865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9ddb2-8073-4ca8-903a-499e7560f0ae None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:09.669931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35e220-97e1-4640-a96a-47e3bd9decbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:09.670824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35e220-97e1-4640-a96a-47e3bd9decbc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:09.671100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35e220-97e1-4640-a96a-47e3bd9decbc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:09.671371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35e220-97e1-4640-a96a-47e3bd9decbc None admin] PATH_INFO: `/v3/projects/ad18ca11afd74e6daccb96c4ae260526` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:09.672540 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35e220-97e1-4640-a96a-47e3bd9decbc None admin] RBAC: Authorizing `identity:get_project(project_id=ad18ca11afd74e6daccb96c4ae260526)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:09.673979 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35e220-97e1-4640-a96a-47e3bd9decbc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:09.675684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 565/1116] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:14:09 2022] GET /identity/v3/projects/ad18ca11afd74e6daccb96c4ae260526 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:14:09.684552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d2eba-552f-4665-b466-c54d5f942788 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:09.788953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d2eba-552f-4665-b466-c54d5f942788 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:09.827865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc8d31-8bd6-4802-a9df-278c94620f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:09.828515 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc8d31-8bd6-4802-a9df-278c94620f97 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:09.828799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc8d31-8bd6-4802-a9df-278c94620f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:09.829077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc8d31-8bd6-4802-a9df-278c94620f97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:09.829910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc8d31-8bd6-4802-a9df-278c94620f97 None admin] RBAC: Adding query filter params (role.id=f4461b279e86422b83e29521e7ca29d0, scope.project.id=ad18ca11afd74e6daccb96c4ae260526, user.id=d98136f09cce4289be4ef67b3efc34b5) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:09.830221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc8d31-8bd6-4802-a9df-278c94620f97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:09.831691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc8d31-8bd6-4802-a9df-278c94620f97 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:09.833499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:09.838520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 552/1117] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:14:09 2022] GET /identity/v3/role_assignments?user.id=d98136f09cce4289be4ef67b3efc34b5&scope.project.id=ad18ca11afd74e6daccb96c4ae260526&role.id=f4461b279e86422b83e29521e7ca29d0 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:12.116771 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cbc6a-5e16-4233-80c2-3cb72ed97f53 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:12.117336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cbc6a-5e16-4233-80c2-3cb72ed97f53 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:12.117336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cbc6a-5e16-4233-80c2-3cb72ed97f53 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:12.119027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 566/1118] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:14:12.124865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a9f40-4be4-4573-b0cc-cf12c31fc53a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:12.125585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a9f40-4be4-4573-b0cc-cf12c31fc53a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:12.125585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a9f40-4be4-4573-b0cc-cf12c31fc53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:12.142514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-ef9a9f40-4be4-4573-b0cc-cf12c31fc53a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:12.143762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ef9a9f40-4be4-4573-b0cc-cf12c31fc53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:12.223711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 553/1119] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:14:12.248294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cbc6a-5e16-4233-80c2-3cb72ed97f53 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:12.313217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:12.381043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cbc6a-5e16-4233-80c2-3cb72ed97f53 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:12.430392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecf4ca-abd5-49ba-98fa-677b71e579fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:12.431780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecf4ca-abd5-49ba-98fa-677b71e579fe None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:12.432225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecf4ca-abd5-49ba-98fa-677b71e579fe None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:12.432672 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ecf4ca-abd5-49ba-98fa-677b71e579fe None admin] PATH_INFO: `/v3/roles/f4461b279e86422b83e29521e7ca29d0` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:12.434515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ecf4ca-abd5-49ba-98fa-677b71e579fe None admin] RBAC: Authorizing `identity:get_role(role_id=f4461b279e86422b83e29521e7ca29d0)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:12.435936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ecf4ca-abd5-49ba-98fa-677b71e579fe None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:12.437340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 567/1120] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:12 2022] GET /identity/v3/roles/f4461b279e86422b83e29521e7ca29d0 => generated 223 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:12.443379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a9f40-4be4-4573-b0cc-cf12c31fc53a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:12.560048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a9f40-4be4-4573-b0cc-cf12c31fc53a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:12.589048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:12.597393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab1ea1-e2bc-477e-a6be-34e910bcf344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:12.598268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab1ea1-e2bc-477e-a6be-34e910bcf344 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:12.598575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab1ea1-e2bc-477e-a6be-34e910bcf344 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:12.598860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab1ea1-e2bc-477e-a6be-34e910bcf344 None admin] PATH_INFO: `/v3/users/d98136f09cce4289be4ef67b3efc34b5` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:12.600082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab1ea1-e2bc-477e-a6be-34e910bcf344 None admin] RBAC: Authorizing `identity:get_user(user_id=d98136f09cce4289be4ef67b3efc34b5)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:12.602107 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab1ea1-e2bc-477e-a6be-34e910bcf344 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:12.603807 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 554/1121] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:12 2022] GET /identity/v3/users/d98136f09cce4289be4ef67b3efc34b5 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:12.609784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecf4ca-abd5-49ba-98fa-677b71e579fe None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:12.719020 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ecf4ca-abd5-49ba-98fa-677b71e579fe None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:12.756210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:12.774711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47619aca-2024-4252-9328-fc3ac46818e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:12.775584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47619aca-2024-4252-9328-fc3ac46818e5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:12.775868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47619aca-2024-4252-9328-fc3ac46818e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:12.776143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47619aca-2024-4252-9328-fc3ac46818e5 None admin] PATH_INFO: `/v3/projects/ad18ca11afd74e6daccb96c4ae260526` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:12.777353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47619aca-2024-4252-9328-fc3ac46818e5 None admin] RBAC: Authorizing `identity:get_project(project_id=ad18ca11afd74e6daccb96c4ae260526)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:12.778847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47619aca-2024-4252-9328-fc3ac46818e5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:12.780581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 568/1122] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:14:12 2022] GET /identity/v3/projects/ad18ca11afd74e6daccb96c4ae260526 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:14:12.787008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab1ea1-e2bc-477e-a6be-34e910bcf344 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:12.920214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab1ea1-e2bc-477e-a6be-34e910bcf344 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:12.960836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f06c06d-3ae1-403d-a3fd-c23be58e4981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:12.961618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06c06d-3ae1-403d-a3fd-c23be58e4981 None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:12.961899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06c06d-3ae1-403d-a3fd-c23be58e4981 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:12.962174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06c06d-3ae1-403d-a3fd-c23be58e4981 None admin] PATH_INFO: `/v3/projects/ad18ca11afd74e6daccb96c4ae260526/users/d98136f09cce4289be4ef67b3efc34b5/roles/f4461b279e86422b83e29521e7ca29d0` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:12.964438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06c06d-3ae1-403d-a3fd-c23be58e4981 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad18ca11afd74e6daccb96c4ae260526, user_id=d98136f09cce4289be4ef67b3efc34b5, role_id=f4461b279e86422b83e29521e7ca29d0)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:12.965893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f06c06d-3ae1-403d-a3fd-c23be58e4981 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:12.982354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 555/1123] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:14:12 2022] PUT /identity/v3/projects/ad18ca11afd74e6daccb96c4ae260526/users/d98136f09cce4289be4ef67b3efc34b5/roles/f4461b279e86422b83e29521e7ca29d0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:14:15.352038 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ca396-5a43-4eb7-b21e-8f1b5984acb7 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:15.352717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ca396-5a43-4eb7-b21e-8f1b5984acb7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:15.352717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ca396-5a43-4eb7-b21e-8f1b5984acb7 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:15.354024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 569/1124] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:15.360339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389eb5f6-1df7-4787-a2e5-1383cd057ea8 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:15.360837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389eb5f6-1df7-4787-a2e5-1383cd057ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:15.360930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389eb5f6-1df7-4787-a2e5-1383cd057ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:15.377962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-389eb5f6-1df7-4787-a2e5-1383cd057ea8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:15.379205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-389eb5f6-1df7-4787-a2e5-1383cd057ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:15.431912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:15.450354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 556/1125] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:15.474486 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ca396-5a43-4eb7-b21e-8f1b5984acb7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:15.584537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ca396-5a43-4eb7-b21e-8f1b5984acb7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:15.619312 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8288f-133c-4842-b36d-bb62f539cf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:15.619966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8288f-133c-4842-b36d-bb62f539cf3e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:15.620246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8288f-133c-4842-b36d-bb62f539cf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:15.620513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8288f-133c-4842-b36d-bb62f539cf3e None admin] PATH_INFO: `/v3/roles/f4461b279e86422b83e29521e7ca29d0` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:15.622173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8288f-133c-4842-b36d-bb62f539cf3e None admin] RBAC: Authorizing `identity:get_role(role_id=f4461b279e86422b83e29521e7ca29d0)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:15.623383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8288f-133c-4842-b36d-bb62f539cf3e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:15.624774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 570/1126] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:15 2022] GET /identity/v3/roles/f4461b279e86422b83e29521e7ca29d0 => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:15.630902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389eb5f6-1df7-4787-a2e5-1383cd057ea8 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:15.737034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389eb5f6-1df7-4787-a2e5-1383cd057ea8 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:15.777515 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2e51-a9ad-4bfa-9db2-5198ee10a768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:15.778382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2e51-a9ad-4bfa-9db2-5198ee10a768 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:15.778691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2e51-a9ad-4bfa-9db2-5198ee10a768 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:15.778966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d2e51-a9ad-4bfa-9db2-5198ee10a768 None admin] PATH_INFO: `/v3/users/d98136f09cce4289be4ef67b3efc34b5` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:15.780158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d2e51-a9ad-4bfa-9db2-5198ee10a768 None admin] RBAC: Authorizing `identity:get_user(user_id=d98136f09cce4289be4ef67b3efc34b5)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:15.781597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d2e51-a9ad-4bfa-9db2-5198ee10a768 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:15.783264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 557/1127] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:15 2022] GET /identity/v3/users/d98136f09cce4289be4ef67b3efc34b5 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:15.789366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8288f-133c-4842-b36d-bb62f539cf3e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:15.854739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:15.899744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8288f-133c-4842-b36d-bb62f539cf3e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:15.944364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6347e8e-42cb-4155-a5c5-6d157ca1a33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:15.945898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6347e8e-42cb-4155-a5c5-6d157ca1a33c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:15.946180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6347e8e-42cb-4155-a5c5-6d157ca1a33c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:15.946455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6347e8e-42cb-4155-a5c5-6d157ca1a33c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:16.014868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6347e8e-42cb-4155-a5c5-6d157ca1a33c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:16.016334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6347e8e-42cb-4155-a5c5-6d157ca1a33c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:16.085751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:16.129668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 571/1128] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:14:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:14:16.135853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2e51-a9ad-4bfa-9db2-5198ee10a768 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:16.199171 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:16.245132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d2e51-a9ad-4bfa-9db2-5198ee10a768 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:16.282602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58742d1-e93c-4449-814b-b831d65f9b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:16.283470 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58742d1-e93c-4449-814b-b831d65f9b78 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:16.283746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58742d1-e93c-4449-814b-b831d65f9b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:16.284019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58742d1-e93c-4449-814b-b831d65f9b78 None admin] PATH_INFO: `/v3/projects/ad18ca11afd74e6daccb96c4ae260526` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:16.285209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58742d1-e93c-4449-814b-b831d65f9b78 None admin] RBAC: Authorizing `identity:get_project(project_id=ad18ca11afd74e6daccb96c4ae260526)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:16.286938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58742d1-e93c-4449-814b-b831d65f9b78 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:16.289184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 558/1129] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:14:16 2022] GET /identity/v3/projects/ad18ca11afd74e6daccb96c4ae260526 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:16.295844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6347e8e-42cb-4155-a5c5-6d157ca1a33c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:16.405223 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6347e8e-42cb-4155-a5c5-6d157ca1a33c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:16.443277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59088733-5b0e-45d0-bb6f-037f3d2ba066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:16.443944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088733-5b0e-45d0-bb6f-037f3d2ba066 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:16.444225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088733-5b0e-45d0-bb6f-037f3d2ba066 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:16.444492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59088733-5b0e-45d0-bb6f-037f3d2ba066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:16.445428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59088733-5b0e-45d0-bb6f-037f3d2ba066 None admin] RBAC: Adding query filter params (role.id=f4461b279e86422b83e29521e7ca29d0, scope.project.id=ad18ca11afd74e6daccb96c4ae260526, user.id=d98136f09cce4289be4ef67b3efc34b5) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:16.445740 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59088733-5b0e-45d0-bb6f-037f3d2ba066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:16.447244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59088733-5b0e-45d0-bb6f-037f3d2ba066 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:16.453747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 572/1130] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:14:16 2022] GET /identity/v3/role_assignments?user.id=d98136f09cce4289be4ef67b3efc34b5&scope.project.id=ad18ca11afd74e6daccb96c4ae260526&role.id=f4461b279e86422b83e29521e7ca29d0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:18.804437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1a1b7-325c-4ee3-8474-b7850f7eec52 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:18.804971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1a1b7-325c-4ee3-8474-b7850f7eec52 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:18.805090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e1a1b7-325c-4ee3-8474-b7850f7eec52 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:18.806429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 559/1131] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:18.811947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcfb92-b793-4551-a8c0-9e02c382226d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:18.812361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcfb92-b793-4551-a8c0-9e02c382226d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:18.812497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcfb92-b793-4551-a8c0-9e02c382226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:18.830219 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7dfcfb92-b793-4551-a8c0-9e02c382226d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:18.831446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7dfcfb92-b793-4551-a8c0-9e02c382226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:18.849717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:18.905780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 573/1132] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:18.930666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1a1b7-325c-4ee3-8474-b7850f7eec52 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:19.037601 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e1a1b7-325c-4ee3-8474-b7850f7eec52 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:19.068382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:19.081329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b284897-68e5-400e-9281-6e0190aaa2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:19.082126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b284897-68e5-400e-9281-6e0190aaa2fd None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:19.082410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b284897-68e5-400e-9281-6e0190aaa2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:19.082720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b284897-68e5-400e-9281-6e0190aaa2fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:19.083952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b284897-68e5-400e-9281-6e0190aaa2fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:19.085348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b284897-68e5-400e-9281-6e0190aaa2fd None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:19.087047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 560/1133] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:14:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:19.093096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcfb92-b793-4551-a8c0-9e02c382226d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:19.196304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcfb92-b793-4551-a8c0-9e02c382226d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:19.232768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bb702-bd5b-4d53-9af9-c759668d6721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:19.233452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb702-bd5b-4d53-9af9-c759668d6721 None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:19.233758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb702-bd5b-4d53-9af9-c759668d6721 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:19.234054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bb702-bd5b-4d53-9af9-c759668d6721 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:19.234893 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bb702-bd5b-4d53-9af9-c759668d6721 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:19.236403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bb702-bd5b-4d53-9af9-c759668d6721 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:19.264596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 574/1134] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:14:19 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:14:21.651741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7b52-0d8b-43be-826a-241edc7e03a2 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:21.652367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7b52-0d8b-43be-826a-241edc7e03a2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:21.652474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f7b52-0d8b-43be-826a-241edc7e03a2 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:21.654945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 561/1135] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:21.663299 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c217-c6cb-48af-a1e8-7b80684df89f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:21.663857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c217-c6cb-48af-a1e8-7b80684df89f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:21.663857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c217-c6cb-48af-a1e8-7b80684df89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:21.671228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:21.684373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9999c217-c6cb-48af-a1e8-7b80684df89f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:21.685584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9999c217-c6cb-48af-a1e8-7b80684df89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:21.761822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 575/1136] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:14:21.786786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7b52-0d8b-43be-826a-241edc7e03a2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:21.896367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f7b52-0d8b-43be-826a-241edc7e03a2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:21.934730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dea193-81e5-499b-abd6-fd51bb1efc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:21.935421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea193-81e5-499b-abd6-fd51bb1efc0c None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:21.935701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea193-81e5-499b-abd6-fd51bb1efc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:21.935974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea193-81e5-499b-abd6-fd51bb1efc0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:21.937200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dea193-81e5-499b-abd6-fd51bb1efc0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:21.938732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dea193-81e5-499b-abd6-fd51bb1efc0c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:21.940465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 562/1137] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:14:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:21.948298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999c217-c6cb-48af-a1e8-7b80684df89f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:22.051478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999c217-c6cb-48af-a1e8-7b80684df89f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:22.086799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ac8f4-1168-4a0b-8828-f265984cf24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:22.087463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ac8f4-1168-4a0b-8828-f265984cf24a None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:22.087742 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ac8f4-1168-4a0b-8828-f265984cf24a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:22.088012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ac8f4-1168-4a0b-8828-f265984cf24a None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:22.088715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ac8f4-1168-4a0b-8828-f265984cf24a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:22.090160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ac8f4-1168-4a0b-8828-f265984cf24a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:22.096940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:22.112287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 576/1138] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:14:21 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:14:24.586766 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fa8e2-7d85-478b-b4b7-e6565db1cf43 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:24.587357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fa8e2-7d85-478b-b4b7-e6565db1cf43 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:24.587357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fa8e2-7d85-478b-b4b7-e6565db1cf43 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:24.588753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 563/1139] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:24.594977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff4968-59b8-4991-95b5-0516d38e5290 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:24.595528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff4968-59b8-4991-95b5-0516d38e5290 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:24.595528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff4968-59b8-4991-95b5-0516d38e5290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:24.618060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c9ff4968-59b8-4991-95b5-0516d38e5290 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:24.619499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c9ff4968-59b8-4991-95b5-0516d38e5290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:24.697008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 577/1140] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:24.721940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fa8e2-7d85-478b-b4b7-e6565db1cf43 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:24.804721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:24.859175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fa8e2-7d85-478b-b4b7-e6565db1cf43 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:24.896839 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97312dc-bd63-4d0e-aace-b03fe0c6226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:24.898269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97312dc-bd63-4d0e-aace-b03fe0c6226d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:24.898573 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97312dc-bd63-4d0e-aace-b03fe0c6226d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:24.898858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97312dc-bd63-4d0e-aace-b03fe0c6226d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:24.910175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97312dc-bd63-4d0e-aace-b03fe0c6226d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-e97312dc-bd63-4d0e-aace-b03fe0c6226d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:14:24.918823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:24.920242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:14:24.921660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 564/1141] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:14:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:14:24.927307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff4968-59b8-4991-95b5-0516d38e5290 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:25.047871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff4968-59b8-4991-95b5-0516d38e5290 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:25.070097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:25.086242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1f3fb-6a60-47ba-a47f-96576922b1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:25.086959 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1f3fb-6a60-47ba-a47f-96576922b1df None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:25.087238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1f3fb-6a60-47ba-a47f-96576922b1df None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:25.087539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1f3fb-6a60-47ba-a47f-96576922b1df None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:25.088300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1f3fb-6a60-47ba-a47f-96576922b1df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:25.088608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1f3fb-6a60-47ba-a47f-96576922b1df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:25.090138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1f3fb-6a60-47ba-a47f-96576922b1df None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:25.099311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 578/1142] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:14:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:25.105843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97312dc-bd63-4d0e-aace-b03fe0c6226d None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:25.223658 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97312dc-bd63-4d0e-aace-b03fe0c6226d None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:25.244943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:25.267162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b393d4-01fd-4005-8f83-3cc4f6263193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:25.268057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b393d4-01fd-4005-8f83-3cc4f6263193 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:25.268340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b393d4-01fd-4005-8f83-3cc4f6263193 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:25.268616 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b393d4-01fd-4005-8f83-3cc4f6263193 None admin] PATH_INFO: `/v3/users/d105922dccc1465eb6d7580d5973c298` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:25.290692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b393d4-01fd-4005-8f83-3cc4f6263193 None admin] RBAC: Authorizing `identity:get_user(user_id=d105922dccc1465eb6d7580d5973c298)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:25.293055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b393d4-01fd-4005-8f83-3cc4f6263193 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:25.296091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 565/1143] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:25 2022] GET /identity/v3/users/d105922dccc1465eb6d7580d5973c298 => generated 286 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:25.302021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1f3fb-6a60-47ba-a47f-96576922b1df None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:25.447278 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1f3fb-6a60-47ba-a47f-96576922b1df None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:25.493166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a2f92-14be-41cf-ad5d-54782c9cedd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:25.494153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a2f92-14be-41cf-ad5d-54782c9cedd5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:25.494691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a2f92-14be-41cf-ad5d-54782c9cedd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:25.495196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a2f92-14be-41cf-ad5d-54782c9cedd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:25.569373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:25.589027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a2f92-14be-41cf-ad5d-54782c9cedd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:25.590505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a2f92-14be-41cf-ad5d-54782c9cedd5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:25.700562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 579/1144] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:14:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 15:14:25.706604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b393d4-01fd-4005-8f83-3cc4f6263193 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:25.816669 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b393d4-01fd-4005-8f83-3cc4f6263193 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:25.851742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217b36f-e725-406c-a8c4-d41120f58fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:25.852606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217b36f-e725-406c-a8c4-d41120f58fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:25.852888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217b36f-e725-406c-a8c4-d41120f58fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:25.853162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217b36f-e725-406c-a8c4-d41120f58fa0 None admin] PATH_INFO: `/v3/projects/edcfcd0576174598a73f28912ecc6392` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:25.854361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217b36f-e725-406c-a8c4-d41120f58fa0 None admin] RBAC: Authorizing `identity:get_project(project_id=edcfcd0576174598a73f28912ecc6392)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:25.855840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217b36f-e725-406c-a8c4-d41120f58fa0 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:25.857572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 566/1145] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:14:25 2022] GET /identity/v3/projects/edcfcd0576174598a73f28912ecc6392 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:25.864444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a2f92-14be-41cf-ad5d-54782c9cedd5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:25.948281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:25.968076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a2f92-14be-41cf-ad5d-54782c9cedd5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:26.002722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee6469e-cf21-4f44-af28-e151de56f24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:26.003370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6469e-cf21-4f44-af28-e151de56f24b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:26.003648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6469e-cf21-4f44-af28-e151de56f24b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:26.003917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee6469e-cf21-4f44-af28-e151de56f24b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:26.004743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6469e-cf21-4f44-af28-e151de56f24b None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=edcfcd0576174598a73f28912ecc6392, user.id=d105922dccc1465eb6d7580d5973c298) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:26.005043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6469e-cf21-4f44-af28-e151de56f24b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:26.006474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee6469e-cf21-4f44-af28-e151de56f24b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:26.012274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 580/1146] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:14:25 2022] GET /identity/v3/role_assignments?user.id=d105922dccc1465eb6d7580d5973c298&scope.project.id=edcfcd0576174598a73f28912ecc6392&role.id=d67707782358417ba9f25b2793ca2890 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:28.322289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b148a8-7b77-46e2-bc02-047476493b19 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:28.322890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b148a8-7b77-46e2-bc02-047476493b19 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:28.322890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b148a8-7b77-46e2-bc02-047476493b19 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:28.324232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 567/1147] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:28.329545 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757f3cf-7f05-450e-8211-8181346bd726 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:28.329938 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757f3cf-7f05-450e-8211-8181346bd726 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:28.330102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757f3cf-7f05-450e-8211-8181346bd726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:28.348057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d757f3cf-7f05-450e-8211-8181346bd726 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:28.349238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d757f3cf-7f05-450e-8211-8181346bd726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:28.422234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 581/1148] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:28.451246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b148a8-7b77-46e2-bc02-047476493b19 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:28.499854 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:28.556187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b148a8-7b77-46e2-bc02-047476493b19 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:28.591541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc2d31-365e-4837-b226-58754fd48ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:28.592240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc2d31-365e-4837-b226-58754fd48ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:28.592525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc2d31-365e-4837-b226-58754fd48ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:28.592800 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc2d31-365e-4837-b226-58754fd48ab0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:28.608207 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc2d31-365e-4837-b226-58754fd48ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-a0bc2d31-365e-4837-b226-58754fd48ab0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:14:28.616682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:28.617922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:14:28.618954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 568/1149] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:14:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:14:28.625974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757f3cf-7f05-450e-8211-8181346bd726 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:28.735066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:28.774190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757f3cf-7f05-450e-8211-8181346bd726 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:28.831128 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea1cd6-4cfb-42fa-9db3-f84d7e139dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:28.832224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea1cd6-4cfb-42fa-9db3-f84d7e139dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:28.832703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea1cd6-4cfb-42fa-9db3-f84d7e139dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:28.833178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea1cd6-4cfb-42fa-9db3-f84d7e139dc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:28.834519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea1cd6-4cfb-42fa-9db3-f84d7e139dc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:28.835107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea1cd6-4cfb-42fa-9db3-f84d7e139dc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:28.837573 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea1cd6-4cfb-42fa-9db3-f84d7e139dc8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:28.851909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 582/1150] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:14:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:28.858965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc2d31-365e-4837-b226-58754fd48ab0 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:28.977699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc2d31-365e-4837-b226-58754fd48ab0 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:28.980106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:29.018616 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e755ee-725f-4640-93af-c06b313cfc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:29.020000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e755ee-725f-4640-93af-c06b313cfc28 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:29.020478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e755ee-725f-4640-93af-c06b313cfc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:29.020953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e755ee-725f-4640-93af-c06b313cfc28 None admin] PATH_INFO: `/v3/users/d105922dccc1465eb6d7580d5973c298` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:29.023315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e755ee-725f-4640-93af-c06b313cfc28 None admin] RBAC: Authorizing `identity:get_user(user_id=d105922dccc1465eb6d7580d5973c298)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:29.025954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e755ee-725f-4640-93af-c06b313cfc28 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:29.029124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 569/1151] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:28 2022] GET /identity/v3/users/d105922dccc1465eb6d7580d5973c298 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:29.039248 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea1cd6-4cfb-42fa-9db3-f84d7e139dc8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:29.145823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea1cd6-4cfb-42fa-9db3-f84d7e139dc8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:29.183331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4cac01-0158-4bbc-b9cc-535bb9499e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:29.184194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cac01-0158-4bbc-b9cc-535bb9499e1b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:29.184478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cac01-0158-4bbc-b9cc-535bb9499e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:29.184748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cac01-0158-4bbc-b9cc-535bb9499e1b None admin] PATH_INFO: `/v3/projects/edcfcd0576174598a73f28912ecc6392` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:29.185928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4cac01-0158-4bbc-b9cc-535bb9499e1b None admin] RBAC: Authorizing `identity:get_project(project_id=edcfcd0576174598a73f28912ecc6392)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:29.187405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4cac01-0158-4bbc-b9cc-535bb9499e1b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:29.189095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 583/1152] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:14:29 2022] GET /identity/v3/projects/edcfcd0576174598a73f28912ecc6392 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:14:29.195387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e755ee-725f-4640-93af-c06b313cfc28 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:29.317882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e755ee-725f-4640-93af-c06b313cfc28 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:29.369118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248c19c-d2e4-494c-a0ff-ee99b9c863dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:29.370513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c19c-d2e4-494c-a0ff-ee99b9c863dd None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:29.370951 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c19c-d2e4-494c-a0ff-ee99b9c863dd None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:29.371343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c19c-d2e4-494c-a0ff-ee99b9c863dd None admin] PATH_INFO: `/v3/projects/edcfcd0576174598a73f28912ecc6392/users/d105922dccc1465eb6d7580d5973c298/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:29.374265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248c19c-d2e4-494c-a0ff-ee99b9c863dd None admin] RBAC: Authorizing `identity:create_grant(project_id=edcfcd0576174598a73f28912ecc6392, user_id=d105922dccc1465eb6d7580d5973c298, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:29.376476 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248c19c-d2e4-494c-a0ff-ee99b9c863dd None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:29.395637 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 570/1153] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:14:29 2022] PUT /identity/v3/projects/edcfcd0576174598a73f28912ecc6392/users/d105922dccc1465eb6d7580d5973c298/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:14:31.764458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b73306-5040-4084-b85a-910c6acaafa9 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:31.765051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b73306-5040-4084-b85a-910c6acaafa9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:31.765368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b73306-5040-4084-b85a-910c6acaafa9 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:31.767684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 584/1154] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:31.775969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9efb0-89c9-4b74-8beb-0e11387baa8e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:31.776633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9efb0-89c9-4b74-8beb-0e11387baa8e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:31.776633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9efb0-89c9-4b74-8beb-0e11387baa8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:31.796544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-54f9efb0-89c9-4b74-8beb-0e11387baa8e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:31.798885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-54f9efb0-89c9-4b74-8beb-0e11387baa8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:31.830181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:31.878762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 571/1155] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:31.922161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b73306-5040-4084-b85a-910c6acaafa9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:31.936091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:32.049381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b73306-5040-4084-b85a-910c6acaafa9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:32.087783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d6403-da9f-46f1-acde-29bfa2c6b005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:32.088578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d6403-da9f-46f1-acde-29bfa2c6b005 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:32.088861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d6403-da9f-46f1-acde-29bfa2c6b005 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:32.089133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d6403-da9f-46f1-acde-29bfa2c6b005 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:32.101928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d6403-da9f-46f1-acde-29bfa2c6b005 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-484d6403-da9f-46f1-acde-29bfa2c6b005 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:14:32.109704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:32.111753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:14:32.113802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 585/1156] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:14:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:14:32.117880 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9efb0-89c9-4b74-8beb-0e11387baa8e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:32.256363 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9efb0-89c9-4b74-8beb-0e11387baa8e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:32.307037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db7950-c921-4a62-bb5f-96ab327c4b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:32.307943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db7950-c921-4a62-bb5f-96ab327c4b56 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:32.308384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db7950-c921-4a62-bb5f-96ab327c4b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:32.308785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db7950-c921-4a62-bb5f-96ab327c4b56 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:32.309894 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db7950-c921-4a62-bb5f-96ab327c4b56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:32.310259 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db7950-c921-4a62-bb5f-96ab327c4b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:32.312046 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db7950-c921-4a62-bb5f-96ab327c4b56 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:32.322764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 572/1157] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:32.332233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d6403-da9f-46f1-acde-29bfa2c6b005 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:32.406613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:32.461283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d6403-da9f-46f1-acde-29bfa2c6b005 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:32.505596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacc33b-45cb-4576-9619-aa6bec11876a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:32.507144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacc33b-45cb-4576-9619-aa6bec11876a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:32.507369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacc33b-45cb-4576-9619-aa6bec11876a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:32.507644 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacc33b-45cb-4576-9619-aa6bec11876a None admin] PATH_INFO: `/v3/users/d105922dccc1465eb6d7580d5973c298` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:32.508916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebacc33b-45cb-4576-9619-aa6bec11876a None admin] RBAC: Authorizing `identity:get_user(user_id=d105922dccc1465eb6d7580d5973c298)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:32.510387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebacc33b-45cb-4576-9619-aa6bec11876a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:32.512091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 586/1158] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:32 2022] GET /identity/v3/users/d105922dccc1465eb6d7580d5973c298 => generated 286 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:32.521583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db7950-c921-4a62-bb5f-96ab327c4b56 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:32.556450 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:32.644025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db7950-c921-4a62-bb5f-96ab327c4b56 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:32.685631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9bf04-4cc6-4386-98e8-391749391023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:32.686313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9bf04-4cc6-4386-98e8-391749391023 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:32.686602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9bf04-4cc6-4386-98e8-391749391023 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:32.687031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9bf04-4cc6-4386-98e8-391749391023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:32.768439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9bf04-4cc6-4386-98e8-391749391023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:32.770024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9bf04-4cc6-4386-98e8-391749391023 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:32.798946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:32.884777 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 573/1159] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:14:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:14:32.891514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacc33b-45cb-4576-9619-aa6bec11876a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:33.004237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacc33b-45cb-4576-9619-aa6bec11876a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:33.041688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e56fff-f8a7-4e3b-a37c-39f66976ba33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:33.042441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e56fff-f8a7-4e3b-a37c-39f66976ba33 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:33.042880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e56fff-f8a7-4e3b-a37c-39f66976ba33 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:33.043158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e56fff-f8a7-4e3b-a37c-39f66976ba33 None admin] PATH_INFO: `/v3/projects/edcfcd0576174598a73f28912ecc6392` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:33.044359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e56fff-f8a7-4e3b-a37c-39f66976ba33 None admin] RBAC: Authorizing `identity:get_project(project_id=edcfcd0576174598a73f28912ecc6392)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:33.045794 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e56fff-f8a7-4e3b-a37c-39f66976ba33 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:33.047508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 587/1160] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:14:32 2022] GET /identity/v3/projects/edcfcd0576174598a73f28912ecc6392 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:33.060679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9bf04-4cc6-4386-98e8-391749391023 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:33.181631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9bf04-4cc6-4386-98e8-391749391023 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:33.211607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:33.223946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d8bdd-2f29-402a-b8f7-5b88d5a32004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:33.224617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d8bdd-2f29-402a-b8f7-5b88d5a32004 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:33.224925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d8bdd-2f29-402a-b8f7-5b88d5a32004 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:33.225208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d8bdd-2f29-402a-b8f7-5b88d5a32004 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:33.226054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d8bdd-2f29-402a-b8f7-5b88d5a32004 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=edcfcd0576174598a73f28912ecc6392, user.id=d105922dccc1465eb6d7580d5973c298) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:33.226372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d8bdd-2f29-402a-b8f7-5b88d5a32004 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:33.227867 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d8bdd-2f29-402a-b8f7-5b88d5a32004 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:33.234640 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 574/1161] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:14:33 2022] GET /identity/v3/role_assignments?user.id=d105922dccc1465eb6d7580d5973c298&scope.project.id=edcfcd0576174598a73f28912ecc6392&role.id=d67707782358417ba9f25b2793ca2890 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:35.618146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086984b1-952a-4de7-b88b-417e9084ee06 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:35.618766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086984b1-952a-4de7-b88b-417e9084ee06 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:35.618766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086984b1-952a-4de7-b88b-417e9084ee06 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:35.620258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 588/1162] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:14:35.626249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8135620-c069-499e-8bc2-01da793e9e3e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:35.626719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8135620-c069-499e-8bc2-01da793e9e3e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:35.626903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8135620-c069-499e-8bc2-01da793e9e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:35.644956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-a8135620-c069-499e-8bc2-01da793e9e3e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:35.646174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a8135620-c069-499e-8bc2-01da793e9e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:35.716970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 575/1163] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:14:35.739242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086984b1-952a-4de7-b88b-417e9084ee06 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:35.742398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:35.849950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086984b1-952a-4de7-b88b-417e9084ee06 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:35.885684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bae28-b319-431f-bb4a-4ff087ea2a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:35.886238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bae28-b319-431f-bb4a-4ff087ea2a87 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:35.886523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bae28-b319-431f-bb4a-4ff087ea2a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:35.886828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bae28-b319-431f-bb4a-4ff087ea2a87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:35.955483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bae28-b319-431f-bb4a-4ff087ea2a87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:35.956957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bae28-b319-431f-bb4a-4ff087ea2a87 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:35.958859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:36.056557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 589/1164] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:14:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:14:36.062603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8135620-c069-499e-8bc2-01da793e9e3e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:36.159592 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8135620-c069-499e-8bc2-01da793e9e3e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:36.177346 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:36.195230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f078d5-a48b-42c8-96c2-da28fccd0a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:36.196036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f078d5-a48b-42c8-96c2-da28fccd0a52 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:36.196318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f078d5-a48b-42c8-96c2-da28fccd0a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:36.196588 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f078d5-a48b-42c8-96c2-da28fccd0a52 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:36.203611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f078d5-a48b-42c8-96c2-da28fccd0a52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:36.205059 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f078d5-a48b-42c8-96c2-da28fccd0a52 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-c7f078d5-a48b-42c8-96c2-da28fccd0a52 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:36.219417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:36.220705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:14:36.222009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 576/1165] 10.222.0.93 () {56 vars in 1117 bytes} [Fri Jun 3 15:14:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:14:36.227311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bae28-b319-431f-bb4a-4ff087ea2a87 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:36.339410 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bae28-b319-431f-bb4a-4ff087ea2a87 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:36.341161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:36.377815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0ce26-7d0c-46a9-bd36-b2898d1e2bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:36.378479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0ce26-7d0c-46a9-bd36-b2898d1e2bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:36.378792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0ce26-7d0c-46a9-bd36-b2898d1e2bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:36.379078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0ce26-7d0c-46a9-bd36-b2898d1e2bd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:36.379819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0ce26-7d0c-46a9-bd36-b2898d1e2bd1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:36.380127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0ce26-7d0c-46a9-bd36-b2898d1e2bd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:36.382388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0ce26-7d0c-46a9-bd36-b2898d1e2bd1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:36.390388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 590/1166] 10.222.0.93 () {56 vars in 1115 bytes} [Fri Jun 3 15:14:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:38.652533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0ac24-159b-4ed6-ab85-1646359c072b None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:38.653084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0ac24-159b-4ed6-ab85-1646359c072b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:38.653084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f0ac24-159b-4ed6-ab85-1646359c072b None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:38.654468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 577/1167] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:38.660178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3713d-fac8-4e4f-9c7f-843b766d0157 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:38.660559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3713d-fac8-4e4f-9c7f-843b766d0157 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:38.660736 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3713d-fac8-4e4f-9c7f-843b766d0157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:38.679456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3cb3713d-fac8-4e4f-9c7f-843b766d0157 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:38.680660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3cb3713d-fac8-4e4f-9c7f-843b766d0157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:38.746165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 591/1168] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:14:38.769601 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0ac24-159b-4ed6-ab85-1646359c072b None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:38.870590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f0ac24-159b-4ed6-ab85-1646359c072b None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:38.903826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ef311-770c-4bc1-a2c6-6f13b437df07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:38.904408 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ef311-770c-4bc1-a2c6-6f13b437df07 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:38.904694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ef311-770c-4bc1-a2c6-6f13b437df07 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:38.904972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ef311-770c-4bc1-a2c6-6f13b437df07 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:38.905721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ef311-770c-4bc1-a2c6-6f13b437df07 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:38.907133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ef311-770c-4bc1-a2c6-6f13b437df07 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:38.921610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 578/1169] 10.222.0.93 () {60 vars in 1138 bytes} [Fri Jun 3 15:14:38 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:14:41.231145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58561626-3584-43c5-bed6-aa14b11f915c None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:41.231914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58561626-3584-43c5-bed6-aa14b11f915c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:41.231914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58561626-3584-43c5-bed6-aa14b11f915c None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:41.233074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 592/1170] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:41.238587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894bd52-dabc-41e3-9fc4-30663cfc60e7 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:41.239070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894bd52-dabc-41e3-9fc4-30663cfc60e7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:41.239166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894bd52-dabc-41e3-9fc4-30663cfc60e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:41.256399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0894bd52-dabc-41e3-9fc4-30663cfc60e7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:41.257590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0894bd52-dabc-41e3-9fc4-30663cfc60e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:41.295868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:41.338367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 579/1171] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:41.363032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58561626-3584-43c5-bed6-aa14b11f915c None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:41.465062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:41.472660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58561626-3584-43c5-bed6-aa14b11f915c None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:41.510773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cedf5-dae7-4486-9991-20bc93acb2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:41.511635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cedf5-dae7-4486-9991-20bc93acb2ad None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:41.511918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cedf5-dae7-4486-9991-20bc93acb2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:41.512194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cedf5-dae7-4486-9991-20bc93acb2ad None admin] PATH_INFO: `/v3/domains/701d567c98b947acbfe01184f4b03e4d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:41.524067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cedf5-dae7-4486-9991-20bc93acb2ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=701d567c98b947acbfe01184f4b03e4d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:41.525555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cedf5-dae7-4486-9991-20bc93acb2ad None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:41.527337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 593/1172] 10.222.0.93 () {56 vars in 1183 bytes} [Fri Jun 3 15:14:41 2022] GET /identity/v3/domains/701d567c98b947acbfe01184f4b03e4d => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:41.535138 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894bd52-dabc-41e3-9fc4-30663cfc60e7 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:41.638681 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894bd52-dabc-41e3-9fc4-30663cfc60e7 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:41.675623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426b46d-019e-4dc3-bc4a-6e59896e4e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:41.676331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426b46d-019e-4dc3-bc4a-6e59896e4e9d None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:41.677378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426b46d-019e-4dc3-bc4a-6e59896e4e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:41.677378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426b46d-019e-4dc3-bc4a-6e59896e4e9d None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:41.677593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426b46d-019e-4dc3-bc4a-6e59896e4e9d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:41.679240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2426b46d-019e-4dc3-bc4a-6e59896e4e9d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:41.707826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 580/1173] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:14:41 2022] POST /identity/v3/projects => generated 361 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:14:44.010227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a67dd-e010-4d6e-8dba-ecded353e1a7 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:44.010799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a67dd-e010-4d6e-8dba-ecded353e1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:44.010916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521a67dd-e010-4d6e-8dba-ecded353e1a7 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:44.012246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 594/1174] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:44.018186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b16cd0-ac43-462a-b3cb-b3dcb02994c8 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:44.018645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b16cd0-ac43-462a-b3cb-b3dcb02994c8 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:44.018918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b16cd0-ac43-462a-b3cb-b3dcb02994c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:44.037301 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-67b16cd0-ac43-462a-b3cb-b3dcb02994c8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:44.038517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-67b16cd0-ac43-462a-b3cb-b3dcb02994c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:44.053392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:44.107363 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 581/1175] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:44.133089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a67dd-e010-4d6e-8dba-ecded353e1a7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:44.238033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521a67dd-e010-4d6e-8dba-ecded353e1a7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:44.273646 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46b9f7b-034c-49ef-8e21-caab1dc62b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:44.274429 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b9f7b-034c-49ef-8e21-caab1dc62b11 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:44.274743 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b9f7b-034c-49ef-8e21-caab1dc62b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:44.275013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46b9f7b-034c-49ef-8e21-caab1dc62b11 None admin] PATH_INFO: `/v3/domains/701d567c98b947acbfe01184f4b03e4d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:44.276189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b9f7b-034c-49ef-8e21-caab1dc62b11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=701d567c98b947acbfe01184f4b03e4d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:44.277629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46b9f7b-034c-49ef-8e21-caab1dc62b11 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:44.279262 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 595/1176] 10.222.0.93 () {56 vars in 1183 bytes} [Fri Jun 3 15:14:44 2022] GET /identity/v3/domains/701d567c98b947acbfe01184f4b03e4d => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:44.285344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b16cd0-ac43-462a-b3cb-b3dcb02994c8 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:44.385139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b16cd0-ac43-462a-b3cb-b3dcb02994c8 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:44.423946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28d03c-2ef0-4630-b666-defd9f79515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:44.424603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28d03c-2ef0-4630-b666-defd9f79515c None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:44.424923 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28d03c-2ef0-4630-b666-defd9f79515c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:44.425243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28d03c-2ef0-4630-b666-defd9f79515c None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:44.425997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28d03c-2ef0-4630-b666-defd9f79515c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:44.428119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28d03c-2ef0-4630-b666-defd9f79515c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:44.453446 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 582/1177] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:14:44 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:14:46.748347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe893f0-0a78-4c39-8a89-e2f8b0ced515 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:46.748904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe893f0-0a78-4c39-8a89-e2f8b0ced515 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:46.748904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe893f0-0a78-4c39-8a89-e2f8b0ced515 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:46.750303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 596/1178] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:46.756395 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8b256-fa79-4db0-9f80-9e6a88c39fe8 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:46.756809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8b256-fa79-4db0-9f80-9e6a88c39fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:46.756948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8b256-fa79-4db0-9f80-9e6a88c39fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:46.768310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:46.774793 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-17e8b256-fa79-4db0-9f80-9e6a88c39fe8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:46.776493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-17e8b256-fa79-4db0-9f80-9e6a88c39fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:46.846328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 583/1179] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:46.871448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe893f0-0a78-4c39-8a89-e2f8b0ced515 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:46.902003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:46.993090 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe893f0-0a78-4c39-8a89-e2f8b0ced515 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:47.027018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6afa1-fb54-4728-9765-0fe72908f9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:47.027802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6afa1-fb54-4728-9765-0fe72908f9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:47.028085 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6afa1-fb54-4728-9765-0fe72908f9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:47.028360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6afa1-fb54-4728-9765-0fe72908f9e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:47.038943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6afa1-fb54-4728-9765-0fe72908f9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-58f6afa1-fb54-4728-9765-0fe72908f9e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:14:47.046661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:47.048459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:14:47.049524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 597/1180] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:14:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:14:47.054829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8b256-fa79-4db0-9f80-9e6a88c39fe8 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:47.158094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8b256-fa79-4db0-9f80-9e6a88c39fe8 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:47.201100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71309c95-a4d2-4651-93e7-0a2f6e25cc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:47.201763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71309c95-a4d2-4651-93e7-0a2f6e25cc72 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:47.202044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71309c95-a4d2-4651-93e7-0a2f6e25cc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:47.202314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71309c95-a4d2-4651-93e7-0a2f6e25cc72 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:47.203087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71309c95-a4d2-4651-93e7-0a2f6e25cc72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:47.203391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71309c95-a4d2-4651-93e7-0a2f6e25cc72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:47.204761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71309c95-a4d2-4651-93e7-0a2f6e25cc72 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:47.207093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:47.214191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 584/1181] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:14:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:47.223094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6afa1-fb54-4728-9765-0fe72908f9e6 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:47.317871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:47.341926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6afa1-fb54-4728-9765-0fe72908f9e6 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:47.379395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205dc31-1280-4049-a905-17360eb86344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:47.380253 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205dc31-1280-4049-a905-17360eb86344 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:47.381145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205dc31-1280-4049-a905-17360eb86344 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:47.381418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205dc31-1280-4049-a905-17360eb86344 None admin] PATH_INFO: `/v3/users/aa5ce730ea01406d908f5a4925e9ec9f` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:47.395799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205dc31-1280-4049-a905-17360eb86344 None admin] RBAC: Authorizing `identity:get_user(user_id=aa5ce730ea01406d908f5a4925e9ec9f)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:47.397396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205dc31-1280-4049-a905-17360eb86344 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:47.399057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 598/1182] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:47 2022] GET /identity/v3/users/aa5ce730ea01406d908f5a4925e9ec9f => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:47.405531 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71309c95-a4d2-4651-93e7-0a2f6e25cc72 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:47.511421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71309c95-a4d2-4651-93e7-0a2f6e25cc72 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:47.548308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f2f83-7f16-49fc-9815-669e327a5656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:47.548949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f2f83-7f16-49fc-9815-669e327a5656 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:47.549229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f2f83-7f16-49fc-9815-669e327a5656 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:47.549503 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f2f83-7f16-49fc-9815-669e327a5656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:47.618899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:47.622960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f2f83-7f16-49fc-9815-669e327a5656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:47.624420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f2f83-7f16-49fc-9815-669e327a5656 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:47.724866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 585/1183] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:14:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:14:47.730875 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205dc31-1280-4049-a905-17360eb86344 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:47.830740 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205dc31-1280-4049-a905-17360eb86344 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:47.865837 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfe330-2332-41b1-8778-d35f5823083f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:47.866718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfe330-2332-41b1-8778-d35f5823083f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:47.867003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfe330-2332-41b1-8778-d35f5823083f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:47.867318 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfe330-2332-41b1-8778-d35f5823083f None admin] PATH_INFO: `/v3/projects/5bdd7245166847e0a6a412f03074e551` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:47.868624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfe330-2332-41b1-8778-d35f5823083f None admin] RBAC: Authorizing `identity:get_project(project_id=5bdd7245166847e0a6a412f03074e551)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:47.870114 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfe330-2332-41b1-8778-d35f5823083f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:47.871785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 599/1184] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:14:47 2022] GET /identity/v3/projects/5bdd7245166847e0a6a412f03074e551 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:47.878505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f2f83-7f16-49fc-9815-669e327a5656 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:47.988068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:47.991643 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f2f83-7f16-49fc-9815-669e327a5656 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:48.025622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05015a-a6c3-4a29-b59e-dabc1ea27111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:48.026270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05015a-a6c3-4a29-b59e-dabc1ea27111 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:48.026578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05015a-a6c3-4a29-b59e-dabc1ea27111 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:48.026861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05015a-a6c3-4a29-b59e-dabc1ea27111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:48.027689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05015a-a6c3-4a29-b59e-dabc1ea27111 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=5bdd7245166847e0a6a412f03074e551, user.id=aa5ce730ea01406d908f5a4925e9ec9f) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:48.028002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05015a-a6c3-4a29-b59e-dabc1ea27111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:48.029441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05015a-a6c3-4a29-b59e-dabc1ea27111 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:48.035044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 586/1185] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:14:47 2022] GET /identity/v3/role_assignments?user.id=aa5ce730ea01406d908f5a4925e9ec9f&scope.project.id=5bdd7245166847e0a6a412f03074e551&role.id=d67707782358417ba9f25b2793ca2890 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:50.323910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cf95c-cac9-47bc-b3df-a2d787a21a1a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:50.324473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cf95c-cac9-47bc-b3df-a2d787a21a1a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:50.324580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cf95c-cac9-47bc-b3df-a2d787a21a1a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:50.325964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 600/1186] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:50.331319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad3b14-3985-4e72-b2fb-2eeacc99574d None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:50.331755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad3b14-3985-4e72-b2fb-2eeacc99574d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:50.331883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad3b14-3985-4e72-b2fb-2eeacc99574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:50.348813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d2ad3b14-3985-4e72-b2fb-2eeacc99574d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:50.350029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d2ad3b14-3985-4e72-b2fb-2eeacc99574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:50.420812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 587/1187] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:50.444674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cf95c-cac9-47bc-b3df-a2d787a21a1a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:50.457457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:50.569098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cf95c-cac9-47bc-b3df-a2d787a21a1a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:50.618774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48b143-e525-4c03-931b-9a7bd2494e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:50.619465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48b143-e525-4c03-931b-9a7bd2494e5a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:50.619745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48b143-e525-4c03-931b-9a7bd2494e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:50.620016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48b143-e525-4c03-931b-9a7bd2494e5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:50.632371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48b143-e525-4c03-931b-9a7bd2494e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-ac48b143-e525-4c03-931b-9a7bd2494e5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:14:50.639696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:50.640814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:14:50.641988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 601/1188] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:14:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:14:50.647141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad3b14-3985-4e72-b2fb-2eeacc99574d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:50.756914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:50.766957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad3b14-3985-4e72-b2fb-2eeacc99574d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:50.817456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd0d58-6756-43ae-8b8f-d7546daf8ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:50.818140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd0d58-6756-43ae-8b8f-d7546daf8ecf None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:50.818419 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd0d58-6756-43ae-8b8f-d7546daf8ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:50.818803 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd0d58-6756-43ae-8b8f-d7546daf8ecf None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:50.819551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd0d58-6756-43ae-8b8f-d7546daf8ecf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:50.819855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd0d58-6756-43ae-8b8f-d7546daf8ecf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:50.821228 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd0d58-6756-43ae-8b8f-d7546daf8ecf None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:50.829353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 588/1189] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:14:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:50.837905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48b143-e525-4c03-931b-9a7bd2494e5a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:50.920461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:50.955418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48b143-e525-4c03-931b-9a7bd2494e5a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:50.993682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f225b-c787-47b6-9c7a-483e27bc0bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:50.994643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f225b-c787-47b6-9c7a-483e27bc0bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:50.995169 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f225b-c787-47b6-9c7a-483e27bc0bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:50.995169 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f225b-c787-47b6-9c7a-483e27bc0bf1 None admin] PATH_INFO: `/v3/users/aa5ce730ea01406d908f5a4925e9ec9f` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:50.996432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f225b-c787-47b6-9c7a-483e27bc0bf1 None admin] RBAC: Authorizing `identity:get_user(user_id=aa5ce730ea01406d908f5a4925e9ec9f)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:50.997793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f225b-c787-47b6-9c7a-483e27bc0bf1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:50.999484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 602/1190] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:50 2022] GET /identity/v3/users/aa5ce730ea01406d908f5a4925e9ec9f => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:14:51.005799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd0d58-6756-43ae-8b8f-d7546daf8ecf None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:51.132942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd0d58-6756-43ae-8b8f-d7546daf8ecf None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:51.178730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875177de-d2e7-40a6-b794-23545843f03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:51.179523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875177de-d2e7-40a6-b794-23545843f03d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:51.179807 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875177de-d2e7-40a6-b794-23545843f03d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:51.180084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875177de-d2e7-40a6-b794-23545843f03d None admin] PATH_INFO: `/v3/projects/5bdd7245166847e0a6a412f03074e551` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:51.181335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875177de-d2e7-40a6-b794-23545843f03d None admin] RBAC: Authorizing `identity:get_project(project_id=5bdd7245166847e0a6a412f03074e551)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:51.182817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875177de-d2e7-40a6-b794-23545843f03d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:51.184577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 589/1191] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:14:51 2022] GET /identity/v3/projects/5bdd7245166847e0a6a412f03074e551 => generated 361 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:51.194913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f225b-c787-47b6-9c7a-483e27bc0bf1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:51.313370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f225b-c787-47b6-9c7a-483e27bc0bf1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:51.349945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd00e9-aed4-434c-aa76-dc973266dd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:51.350853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd00e9-aed4-434c-aa76-dc973266dd3d None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:51.351163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd00e9-aed4-434c-aa76-dc973266dd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:51.351435 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd00e9-aed4-434c-aa76-dc973266dd3d None admin] PATH_INFO: `/v3/projects/5bdd7245166847e0a6a412f03074e551/users/aa5ce730ea01406d908f5a4925e9ec9f/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:51.353453 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd00e9-aed4-434c-aa76-dc973266dd3d None admin] RBAC: Authorizing `identity:create_grant(project_id=5bdd7245166847e0a6a412f03074e551, user_id=aa5ce730ea01406d908f5a4925e9ec9f, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:51.354961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd00e9-aed4-434c-aa76-dc973266dd3d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:51.364100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:51.371168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 603/1192] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:14:51 2022] PUT /identity/v3/projects/5bdd7245166847e0a6a412f03074e551/users/aa5ce730ea01406d908f5a4925e9ec9f/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:14:53.677523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749962b-52d7-4154-8dfd-459476c5032c None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:53.678019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749962b-52d7-4154-8dfd-459476c5032c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:53.678421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749962b-52d7-4154-8dfd-459476c5032c None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:53.680035 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 590/1193] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:14:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:14:53.686578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5216a-69dc-4032-b182-cc8202d7bb06 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:53.687280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5216a-69dc-4032-b182-cc8202d7bb06 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:53.687588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5216a-69dc-4032-b182-cc8202d7bb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:53.704935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a4e5216a-69dc-4032-b182-cc8202d7bb06 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:14:53.706138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a4e5216a-69dc-4032-b182-cc8202d7bb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:14:53.790015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 604/1194] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:14:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:14:53.820336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749962b-52d7-4154-8dfd-459476c5032c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:53.848746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:53.934114 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749962b-52d7-4154-8dfd-459476c5032c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:53.977072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810d05-a3fc-4718-9a24-f29e3c9d1026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:53.977874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810d05-a3fc-4718-9a24-f29e3c9d1026 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:53.978150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810d05-a3fc-4718-9a24-f29e3c9d1026 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:53.978428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810d05-a3fc-4718-9a24-f29e3c9d1026 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:53.989355 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef810d05-a3fc-4718-9a24-f29e3c9d1026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ef810d05-a3fc-4718-9a24-f29e3c9d1026 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:14:53.997011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:14:53.998830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:14:54.001327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 591/1195] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:14:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:14:54.004464 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5216a-69dc-4032-b182-cc8202d7bb06 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:54.126135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5216a-69dc-4032-b182-cc8202d7bb06 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:54.155332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:54.164171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01176cdd-ee9c-4ed4-9fa3-482c9790f6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:54.164843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01176cdd-ee9c-4ed4-9fa3-482c9790f6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:54.165127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01176cdd-ee9c-4ed4-9fa3-482c9790f6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:54.165402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01176cdd-ee9c-4ed4-9fa3-482c9790f6e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:54.166153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01176cdd-ee9c-4ed4-9fa3-482c9790f6e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:54.166458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01176cdd-ee9c-4ed4-9fa3-482c9790f6e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:54.167868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01176cdd-ee9c-4ed4-9fa3-482c9790f6e0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:54.176789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 605/1196] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:14:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:54.184231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810d05-a3fc-4718-9a24-f29e3c9d1026 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:54.275217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:54.294243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810d05-a3fc-4718-9a24-f29e3c9d1026 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:54.332213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64337ec9-47bd-4da5-897c-aefb5a7cf03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:54.333423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64337ec9-47bd-4da5-897c-aefb5a7cf03f None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:54.333786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64337ec9-47bd-4da5-897c-aefb5a7cf03f None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:54.334128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64337ec9-47bd-4da5-897c-aefb5a7cf03f None admin] PATH_INFO: `/v3/users/aa5ce730ea01406d908f5a4925e9ec9f` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:54.336120 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64337ec9-47bd-4da5-897c-aefb5a7cf03f None admin] RBAC: Authorizing `identity:get_user(user_id=aa5ce730ea01406d908f5a4925e9ec9f)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:54.337952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64337ec9-47bd-4da5-897c-aefb5a7cf03f None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:54.340174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 592/1197] 10.222.0.93 () {56 vars in 1177 bytes} [Fri Jun 3 15:14:54 2022] GET /identity/v3/users/aa5ce730ea01406d908f5a4925e9ec9f => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:54.346478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01176cdd-ee9c-4ed4-9fa3-482c9790f6e0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:54.455711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01176cdd-ee9c-4ed4-9fa3-482c9790f6e0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:54.490629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec579-0004-4831-b0f5-cd330422b27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:54.491471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec579-0004-4831-b0f5-cd330422b27c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:54.491970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec579-0004-4831-b0f5-cd330422b27c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:54.492258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec579-0004-4831-b0f5-cd330422b27c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:54.557896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:54.572466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ec579-0004-4831-b0f5-cd330422b27c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:54.575029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ec579-0004-4831-b0f5-cd330422b27c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:54.691850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 606/1198] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:14:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:14:54.697833 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64337ec9-47bd-4da5-897c-aefb5a7cf03f None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:54.808941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64337ec9-47bd-4da5-897c-aefb5a7cf03f None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:54.846073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9882d-1523-4c38-b72f-1465ba878933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:54.847074 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9882d-1523-4c38-b72f-1465ba878933 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:54.847241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9882d-1523-4c38-b72f-1465ba878933 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:54.847512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9882d-1523-4c38-b72f-1465ba878933 None admin] PATH_INFO: `/v3/projects/5bdd7245166847e0a6a412f03074e551` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:54.848756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9882d-1523-4c38-b72f-1465ba878933 None admin] RBAC: Authorizing `identity:get_project(project_id=5bdd7245166847e0a6a412f03074e551)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:54.850203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9882d-1523-4c38-b72f-1465ba878933 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:54.851977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 593/1199] 10.222.0.93 () {56 vars in 1186 bytes} [Fri Jun 3 15:14:54 2022] GET /identity/v3/projects/5bdd7245166847e0a6a412f03074e551 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:14:54.859772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec579-0004-4831-b0f5-cd330422b27c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:14:54.994066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:14:55.003913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec579-0004-4831-b0f5-cd330422b27c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:14:55.038944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f0a0d-0b64-4f04-b98d-ac137c590da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:14:55.039590 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f0a0d-0b64-4f04-b98d-ac137c590da2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:14:55.039869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f0a0d-0b64-4f04-b98d-ac137c590da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:14:55.040144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f0a0d-0b64-4f04-b98d-ac137c590da2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:14:55.040972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f0a0d-0b64-4f04-b98d-ac137c590da2 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=5bdd7245166847e0a6a412f03074e551, user.id=aa5ce730ea01406d908f5a4925e9ec9f) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:14:55.041365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f0a0d-0b64-4f04-b98d-ac137c590da2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:14:55.042844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f0a0d-0b64-4f04-b98d-ac137c590da2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:14:55.048796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 607/1200] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:14:54 2022] GET /identity/v3/role_assignments?user.id=aa5ce730ea01406d908f5a4925e9ec9f&scope.project.id=5bdd7245166847e0a6a412f03074e551&role.id=d67707782358417ba9f25b2793ca2890 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:15:04.841512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=91 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:15:05.046040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=23 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:00.614987 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5af15b-a332-43de-bf50-12f24cfdeb21 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:00.615761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5af15b-a332-43de-bf50-12f24cfdeb21 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:00.615761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5af15b-a332-43de-bf50-12f24cfdeb21 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:00.617740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 594/1201] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:16:00.626521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011aa23-7d5f-4784-bb5e-f15c4396b2b2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:00.627124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011aa23-7d5f-4784-bb5e-f15c4396b2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:00.627438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011aa23-7d5f-4784-bb5e-f15c4396b2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:00.663569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f011aa23-7d5f-4784-bb5e-f15c4396b2b2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:16:00.665602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f011aa23-7d5f-4784-bb5e-f15c4396b2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:16:00.777480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 608/1202] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:16:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:16:00.814574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5af15b-a332-43de-bf50-12f24cfdeb21 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:01.056758 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5af15b-a332-43de-bf50-12f24cfdeb21 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:01.115699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e7a57-d66e-4e79-9662-7facd5b19264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:01.116540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e7a57-d66e-4e79-9662-7facd5b19264 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:01.116828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e7a57-d66e-4e79-9662-7facd5b19264 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:01.117117 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e7a57-d66e-4e79-9662-7facd5b19264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:01.134361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e7a57-d66e-4e79-9662-7facd5b19264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:01.135947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e7a57-d66e-4e79-9662-7facd5b19264 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-345e7a57-d66e-4e79-9662-7facd5b19264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:01.162655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:01.164742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:01.165772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:01.165772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:01.165772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:01.165772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:01.165772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 595/1203] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:16:01.178592 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011aa23-7d5f-4784-bb5e-f15c4396b2b2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:01.373442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011aa23-7d5f-4784-bb5e-f15c4396b2b2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:01.435255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:01.449430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a6f5a-4c13-4a64-a523-24757e01c2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:01.450228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a6f5a-4c13-4a64-a523-24757e01c2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:01.450556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a6f5a-4c13-4a64-a523-24757e01c2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:01.450922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a6f5a-4c13-4a64-a523-24757e01c2c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:01.452097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a6f5a-4c13-4a64-a523-24757e01c2c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:01.452492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a6f5a-4c13-4a64-a523-24757e01c2c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:01.455724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a6f5a-4c13-4a64-a523-24757e01c2c1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:01.480499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 609/1204] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:01.482259 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e7a57-d66e-4e79-9662-7facd5b19264 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:01.584906 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:01.689246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e7a57-d66e-4e79-9662-7facd5b19264 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:01.752009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5625f2-2ba3-48fd-a465-f96009c07b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:01.752749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5625f2-2ba3-48fd-a465-f96009c07b92 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:01.753121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5625f2-2ba3-48fd-a465-f96009c07b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:01.753367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5625f2-2ba3-48fd-a465-f96009c07b92 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:01.754119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5625f2-2ba3-48fd-a465-f96009c07b92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:01.755708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5625f2-2ba3-48fd-a465-f96009c07b92 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:01.786364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 596/1205] 10.222.0.93 () {60 vars in 1132 bytes} [Fri Jun 3 15:16:01 2022] POST /identity/v3/users => generated 251 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:16:04.805427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891c422-a90d-4f33-8989-77572144dfe2 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:04.806345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891c422-a90d-4f33-8989-77572144dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:04.806345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891c422-a90d-4f33-8989-77572144dfe2 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:04.807443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 610/1206] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:16:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:16:04.814203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb2bfb-b371-4ef6-abc6-ac7a7837733b None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:04.814839 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb2bfb-b371-4ef6-abc6-ac7a7837733b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:04.814988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb2bfb-b371-4ef6-abc6-ac7a7837733b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:04.835373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-10cb2bfb-b371-4ef6-abc6-ac7a7837733b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:16:04.836618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-10cb2bfb-b371-4ef6-abc6-ac7a7837733b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:16:04.943917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 597/1207] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:16:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:16:04.971157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891c422-a90d-4f33-8989-77572144dfe2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:05.127207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891c422-a90d-4f33-8989-77572144dfe2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:05.202978 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a3833-6034-4de5-b7cd-58cc7e84a505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:05.204102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a3833-6034-4de5-b7cd-58cc7e84a505 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:05.204977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a3833-6034-4de5-b7cd-58cc7e84a505 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:05.205255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a3833-6034-4de5-b7cd-58cc7e84a505 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:05.218306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a3833-6034-4de5-b7cd-58cc7e84a505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-920a3833-6034-4de5-b7cd-58cc7e84a505 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:16:05.230046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:05.231942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:16:05.233692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:05.238695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 611/1208] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:16:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:05.239917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb2bfb-b371-4ef6-abc6-ac7a7837733b None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:05.307235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:05.413427 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb2bfb-b371-4ef6-abc6-ac7a7837733b None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:05.463350 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46421099-e263-4566-b939-a36832d0afb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:05.463928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46421099-e263-4566-b939-a36832d0afb7 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:05.464209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46421099-e263-4566-b939-a36832d0afb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:05.464568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46421099-e263-4566-b939-a36832d0afb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:05.465381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46421099-e263-4566-b939-a36832d0afb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:05.465680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46421099-e263-4566-b939-a36832d0afb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:05.467549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46421099-e263-4566-b939-a36832d0afb7 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:05.482196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 598/1209] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:16:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:05.490335 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a3833-6034-4de5-b7cd-58cc7e84a505 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:05.580323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:05.674126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a3833-6034-4de5-b7cd-58cc7e84a505 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:05.741716 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aae1fe-3a10-4ae8-af79-c3a5858a81cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:05.742397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aae1fe-3a10-4ae8-af79-c3a5858a81cd None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:05.742707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aae1fe-3a10-4ae8-af79-c3a5858a81cd None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:05.742992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aae1fe-3a10-4ae8-af79-c3a5858a81cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:05.755664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aae1fe-3a10-4ae8-af79-c3a5858a81cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:05.756982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aae1fe-3a10-4ae8-af79-c3a5858a81cd None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-05aae1fe-3a10-4ae8-af79-c3a5858a81cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:05.776533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:05.777685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:05.778668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:05.778668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:05.778668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:05.778668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:05.781171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 612/1210] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:05.788239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46421099-e263-4566-b939-a36832d0afb7 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:05.969621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:05.975280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46421099-e263-4566-b939-a36832d0afb7 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:06.021730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe4b0c-9aca-41a8-8fce-7cc51b230839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:06.022835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe4b0c-9aca-41a8-8fce-7cc51b230839 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:06.023358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe4b0c-9aca-41a8-8fce-7cc51b230839 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:06.023922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe4b0c-9aca-41a8-8fce-7cc51b230839 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:06.025037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe4b0c-9aca-41a8-8fce-7cc51b230839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:06.025576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe4b0c-9aca-41a8-8fce-7cc51b230839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:06.027330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe4b0c-9aca-41a8-8fce-7cc51b230839 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:06.044865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 599/1211] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:06.052424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aae1fe-3a10-4ae8-af79-c3a5858a81cd None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:06.226924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aae1fe-3a10-4ae8-af79-c3a5858a81cd None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:06.229105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:06.293322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a661502-ab95-4259-95e9-f5daad9d8ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:06.294688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a661502-ab95-4259-95e9-f5daad9d8ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:06.295201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a661502-ab95-4259-95e9-f5daad9d8ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:06.295685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a661502-ab95-4259-95e9-f5daad9d8ff7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:06.307764 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a661502-ab95-4259-95e9-f5daad9d8ff7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:06.309398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a661502-ab95-4259-95e9-f5daad9d8ff7 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-0a661502-ab95-4259-95e9-f5daad9d8ff7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:16:06.325670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:06.332420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:06.333416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:06.333416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:06.333416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:16:06.333416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:16:06.333416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:16:06.333416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:16:06.333416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:16:06.333416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:16:06.333416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:06.333416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:06.333416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 613/1212] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:16:06.334068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe4b0c-9aca-41a8-8fce-7cc51b230839 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:06.464317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe4b0c-9aca-41a8-8fce-7cc51b230839 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:06.513858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f177551-0079-484e-a93d-173c57a26e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:06.514911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f177551-0079-484e-a93d-173c57a26e06 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:06.515209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f177551-0079-484e-a93d-173c57a26e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:06.515488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f177551-0079-484e-a93d-173c57a26e06 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:06.516341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f177551-0079-484e-a93d-173c57a26e06 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:06.516719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f177551-0079-484e-a93d-173c57a26e06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:06.518203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f177551-0079-484e-a93d-173c57a26e06 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:06.541599 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 600/1213] 10.222.0.93 () {56 vars in 1143 bytes} [Fri Jun 3 15:16:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:06.548534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a661502-ab95-4259-95e9-f5daad9d8ff7 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:06.711949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a661502-ab95-4259-95e9-f5daad9d8ff7 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:06.756386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fe302-c7fd-42e6-90d5-15271c6ccd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:06.757961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fe302-c7fd-42e6-90d5-15271c6ccd51 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:06.757961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fe302-c7fd-42e6-90d5-15271c6ccd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:06.757961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606fe302-c7fd-42e6-90d5-15271c6ccd51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:06.792844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:06.854045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fe302-c7fd-42e6-90d5-15271c6ccd51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:06.854658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606fe302-c7fd-42e6-90d5-15271c6ccd51 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:06.984271 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 614/1214] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:16:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:16:06.990210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f177551-0079-484e-a93d-173c57a26e06 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:07.019869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:07.131604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f177551-0079-484e-a93d-173c57a26e06 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:07.185034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a821e7-ddc3-4fa1-bcda-befc5725c152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:07.186181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a821e7-ddc3-4fa1-bcda-befc5725c152 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:07.187311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a821e7-ddc3-4fa1-bcda-befc5725c152 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:07.187311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a821e7-ddc3-4fa1-bcda-befc5725c152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:07.202755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a821e7-ddc3-4fa1-bcda-befc5725c152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:07.207441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a821e7-ddc3-4fa1-bcda-befc5725c152 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-f8a821e7-ddc3-4fa1-bcda-befc5725c152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:07.228513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:07.229933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:07.230914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:07.230914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:07.230914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:07.230914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:07.230914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 601/1215] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:07.237683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fe302-c7fd-42e6-90d5-15271c6ccd51 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:07.322289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:07.374071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606fe302-c7fd-42e6-90d5-15271c6ccd51 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:07.421642 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109f623-c865-427a-a7fd-544820c6f461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:07.422232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109f623-c865-427a-a7fd-544820c6f461 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:07.422519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109f623-c865-427a-a7fd-544820c6f461 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:07.423340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109f623-c865-427a-a7fd-544820c6f461 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:07.424429 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109f623-c865-427a-a7fd-544820c6f461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:07.424718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109f623-c865-427a-a7fd-544820c6f461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:07.426135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109f623-c865-427a-a7fd-544820c6f461 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:07.440630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 615/1216] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:07.447653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a821e7-ddc3-4fa1-bcda-befc5725c152 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:07.559473 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:07.612666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a821e7-ddc3-4fa1-bcda-befc5725c152 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:07.680527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f1df0-93b2-4eaf-a253-a36a71cfc293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:07.680527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f1df0-93b2-4eaf-a253-a36a71cfc293 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:07.680806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f1df0-93b2-4eaf-a253-a36a71cfc293 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:07.681171 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f1df0-93b2-4eaf-a253-a36a71cfc293 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:07.691720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f1df0-93b2-4eaf-a253-a36a71cfc293 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:07.693649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f1df0-93b2-4eaf-a253-a36a71cfc293 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ad8f1df0-93b2-4eaf-a253-a36a71cfc293 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:16:07.716224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:07.718280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:07.728621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:16:07.728621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:16:07.728621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:16:07.728621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:16:07.728621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:16:07.728621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:16:07.728621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:07.728621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:07.728621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 602/1217] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:16:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:07.729639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109f623-c865-427a-a7fd-544820c6f461 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:07.900703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109f623-c865-427a-a7fd-544820c6f461 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:07.941765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:07.963263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d075a6-f16c-4505-a634-23c807b8ef07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:07.964364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d075a6-f16c-4505-a634-23c807b8ef07 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:07.964842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d075a6-f16c-4505-a634-23c807b8ef07 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:07.965332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d075a6-f16c-4505-a634-23c807b8ef07 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:07.966817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d075a6-f16c-4505-a634-23c807b8ef07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:07.967353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d075a6-f16c-4505-a634-23c807b8ef07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:07.970818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d075a6-f16c-4505-a634-23c807b8ef07 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:07.994446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 616/1218] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:08.002956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f1df0-93b2-4eaf-a253-a36a71cfc293 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:08.183471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f1df0-93b2-4eaf-a253-a36a71cfc293 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:08.217453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:08.229143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30a6fe-210a-4d79-bdfc-997f577fc0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:08.230109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30a6fe-210a-4d79-bdfc-997f577fc0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:08.230669 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30a6fe-210a-4d79-bdfc-997f577fc0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:08.231068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30a6fe-210a-4d79-bdfc-997f577fc0f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:08.232604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30a6fe-210a-4d79-bdfc-997f577fc0f4 None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=50625c2b38ed41698ba6cccdb12730ef) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:08.233044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30a6fe-210a-4d79-bdfc-997f577fc0f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:08.234519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30a6fe-210a-4d79-bdfc-997f577fc0f4 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:08.240947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 603/1219] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:16:08 2022] GET /identity/v3/role_assignments?user.id=50625c2b38ed41698ba6cccdb12730ef&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:11.321365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56eb108-7f4b-4e95-8f05-b17c127c0144 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:11.322918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56eb108-7f4b-4e95-8f05-b17c127c0144 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:11.322918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56eb108-7f4b-4e95-8f05-b17c127c0144 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:11.323660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 617/1220] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:16:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:16:11.332808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657dfe0-16bb-4f57-901b-8effb1b8358a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:11.332808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657dfe0-16bb-4f57-901b-8effb1b8358a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:11.332808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657dfe0-16bb-4f57-901b-8effb1b8358a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:11.355805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-a657dfe0-16bb-4f57-901b-8effb1b8358a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:16:11.357040 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a657dfe0-16bb-4f57-901b-8effb1b8358a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:16:11.434421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 604/1221] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:16:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:16:11.458398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56eb108-7f4b-4e95-8f05-b17c127c0144 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:11.596038 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56eb108-7f4b-4e95-8f05-b17c127c0144 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:11.659355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbff390-d265-40b2-ab2a-d8007a0531c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:11.660811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff390-d265-40b2-ab2a-d8007a0531c4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:11.661301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff390-d265-40b2-ab2a-d8007a0531c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:11.661792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff390-d265-40b2-ab2a-d8007a0531c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:11.679706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbff390-d265-40b2-ab2a-d8007a0531c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-4fbff390-d265-40b2-ab2a-d8007a0531c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:16:11.688118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:11.690094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:11.691119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 618/1222] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:16:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:16:11.696630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657dfe0-16bb-4f57-901b-8effb1b8358a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:11.856075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657dfe0-16bb-4f57-901b-8effb1b8358a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:11.889973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:11.907971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc218ba6-b87b-4127-a796-3fc70c857601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:11.909136 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc218ba6-b87b-4127-a796-3fc70c857601 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:11.909439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc218ba6-b87b-4127-a796-3fc70c857601 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:11.909728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc218ba6-b87b-4127-a796-3fc70c857601 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:11.910914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc218ba6-b87b-4127-a796-3fc70c857601 None admin] RBAC: Adding query filter params (name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:11.911442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc218ba6-b87b-4127-a796-3fc70c857601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:11.913899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc218ba6-b87b-4127-a796-3fc70c857601 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:11.927348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 605/1223] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:16:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:11.933216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbff390-d265-40b2-ab2a-d8007a0531c4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:11.989643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:12.063549 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbff390-d265-40b2-ab2a-d8007a0531c4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:12.116678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a97a04-7f03-4df3-8dbe-e8a198f13cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:12.118523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a97a04-7f03-4df3-8dbe-e8a198f13cfa None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:12.118847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a97a04-7f03-4df3-8dbe-e8a198f13cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:12.119164 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a97a04-7f03-4df3-8dbe-e8a198f13cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:12.139159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a97a04-7f03-4df3-8dbe-e8a198f13cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:12.141973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a97a04-7f03-4df3-8dbe-e8a198f13cfa None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-64a97a04-7f03-4df3-8dbe-e8a198f13cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:12.166807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:12.167992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:12.169193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:12.169193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:12.169193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:12.169193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:12.170643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 619/1224] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:12.176561 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc218ba6-b87b-4127-a796-3fc70c857601 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:12.307536 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc218ba6-b87b-4127-a796-3fc70c857601 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:12.355517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbc570-adee-41d0-a193-36e7de4521be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:12.356194 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc570-adee-41d0-a193-36e7de4521be None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:12.356474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc570-adee-41d0-a193-36e7de4521be None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:12.356750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfbc570-adee-41d0-a193-36e7de4521be None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:12.357489 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbc570-adee-41d0-a193-36e7de4521be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:12.357800 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbc570-adee-41d0-a193-36e7de4521be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:12.359453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfbc570-adee-41d0-a193-36e7de4521be None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:12.372775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 606/1225] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:12.378951 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a97a04-7f03-4df3-8dbe-e8a198f13cfa None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:12.500612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:12.514022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a97a04-7f03-4df3-8dbe-e8a198f13cfa None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:12.566308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f38faf-3873-4894-9b3d-e077edc9f357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:12.567258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f38faf-3873-4894-9b3d-e077edc9f357 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:12.567585 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f38faf-3873-4894-9b3d-e077edc9f357 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:12.567864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f38faf-3873-4894-9b3d-e077edc9f357 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:12.579737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f38faf-3873-4894-9b3d-e077edc9f357 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:12.582103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f38faf-3873-4894-9b3d-e077edc9f357 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-c5f38faf-3873-4894-9b3d-e077edc9f357 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:16:12.606582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:12.608941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:12.613912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:12.613912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:12.613912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:16:12.613912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:16:12.613912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:16:12.613912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:16:12.613912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:16:12.613912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:16:12.613912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:12.613912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:12.619649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbc570-adee-41d0-a193-36e7de4521be None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:12.620129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 620/1226] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:12.707851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:12.787080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfbc570-adee-41d0-a193-36e7de4521be None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:12.832866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94aee8-b4b6-4d1c-853c-bacceeac4922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:12.834149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94aee8-b4b6-4d1c-853c-bacceeac4922 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:12.834771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94aee8-b4b6-4d1c-853c-bacceeac4922 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:12.835382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94aee8-b4b6-4d1c-853c-bacceeac4922 None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:12.837101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94aee8-b4b6-4d1c-853c-bacceeac4922 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:12.837889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94aee8-b4b6-4d1c-853c-bacceeac4922 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:12.840697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94aee8-b4b6-4d1c-853c-bacceeac4922 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:12.866892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 607/1227] 10.222.0.93 () {56 vars in 1143 bytes} [Fri Jun 3 15:16:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:12.874253 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f38faf-3873-4894-9b3d-e077edc9f357 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:13.020637 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f38faf-3873-4894-9b3d-e077edc9f357 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:13.064735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a24fc-fa1f-4236-a928-7e5e3b97a71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:13.065525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a24fc-fa1f-4236-a928-7e5e3b97a71e None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:13.065801 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a24fc-fa1f-4236-a928-7e5e3b97a71e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:13.066075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a24fc-fa1f-4236-a928-7e5e3b97a71e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:13.082841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a24fc-fa1f-4236-a928-7e5e3b97a71e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:13.084829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a24fc-fa1f-4236-a928-7e5e3b97a71e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-be9a24fc-fa1f-4236-a928-7e5e3b97a71e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:13.114774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:13.116522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:13.118403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:13.118403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:13.118403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:13.118403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:13.118957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 621/1228] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:13.125421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94aee8-b4b6-4d1c-853c-bacceeac4922 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:13.253659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94aee8-b4b6-4d1c-853c-bacceeac4922 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:13.258236 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:13.299940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b723a41f-8ea1-4a1a-aa12-c68acedf43d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:13.300570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723a41f-8ea1-4a1a-aa12-c68acedf43d3 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:13.300893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723a41f-8ea1-4a1a-aa12-c68acedf43d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:13.301210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b723a41f-8ea1-4a1a-aa12-c68acedf43d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:13.301984 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b723a41f-8ea1-4a1a-aa12-c68acedf43d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:13.302331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b723a41f-8ea1-4a1a-aa12-c68acedf43d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:13.303847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b723a41f-8ea1-4a1a-aa12-c68acedf43d3 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:13.315820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 608/1229] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:13.321509 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a24fc-fa1f-4236-a928-7e5e3b97a71e None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:13.331673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:13.439070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a24fc-fa1f-4236-a928-7e5e3b97a71e None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:13.477470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c39dc-46e3-4647-8b22-3ab879c64a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:13.478261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c39dc-46e3-4647-8b22-3ab879c64a8b None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:13.478567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c39dc-46e3-4647-8b22-3ab879c64a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:13.478853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c39dc-46e3-4647-8b22-3ab879c64a8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:13.492162 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c39dc-46e3-4647-8b22-3ab879c64a8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:13.493646 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291c39dc-46e3-4647-8b22-3ab879c64a8b None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-291c39dc-46e3-4647-8b22-3ab879c64a8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:16:13.509125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:13.510434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:13.512002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:16:13.512002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:16:13.512002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:16:13.512002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:16:13.512002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:16:13.512002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:16:13.512002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:13.512002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:13.512002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 622/1230] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:16:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:13.516962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b723a41f-8ea1-4a1a-aa12-c68acedf43d3 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:13.637587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b723a41f-8ea1-4a1a-aa12-c68acedf43d3 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:13.697972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3093513-8f46-466d-bf77-afbc22031b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:13.698686 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3093513-8f46-466d-bf77-afbc22031b0d None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:13.698980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3093513-8f46-466d-bf77-afbc22031b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:13.699262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3093513-8f46-466d-bf77-afbc22031b0d None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:13.700065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3093513-8f46-466d-bf77-afbc22031b0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:13.700377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3093513-8f46-466d-bf77-afbc22031b0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:13.701867 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3093513-8f46-466d-bf77-afbc22031b0d None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:13.716299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 609/1231] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:16:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:13.724209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c39dc-46e3-4647-8b22-3ab879c64a8b None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:13.802687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:13.846416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c39dc-46e3-4647-8b22-3ab879c64a8b None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:13.890703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904eb778-b83a-4497-a8af-6c4b249ab8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:13.891729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904eb778-b83a-4497-a8af-6c4b249ab8d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:13.892100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904eb778-b83a-4497-a8af-6c4b249ab8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:13.892964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904eb778-b83a-4497-a8af-6c4b249ab8d4 None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/50625c2b38ed41698ba6cccdb12730ef/roles/cb4aa5570b15465eb4e81979b1019570` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:13.913631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904eb778-b83a-4497-a8af-6c4b249ab8d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=50625c2b38ed41698ba6cccdb12730ef, role_id=cb4aa5570b15465eb4e81979b1019570)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:13.915817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904eb778-b83a-4497-a8af-6c4b249ab8d4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:13.935556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 623/1232] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:16:13 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/50625c2b38ed41698ba6cccdb12730ef/roles/cb4aa5570b15465eb4e81979b1019570 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:16:16.476225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1321e-cbef-4037-b042-11f3a91ae465 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:16.476940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1321e-cbef-4037-b042-11f3a91ae465 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:16.477188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1321e-cbef-4037-b042-11f3a91ae465 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:16.482576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 610/1233] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:16:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:16:16.485732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734ebc4-1be4-4ff9-a5d8-8307bf690763 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:16.486512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734ebc4-1be4-4ff9-a5d8-8307bf690763 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:16.486512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734ebc4-1be4-4ff9-a5d8-8307bf690763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:16.504824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1734ebc4-1be4-4ff9-a5d8-8307bf690763 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:16:16.506099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1734ebc4-1be4-4ff9-a5d8-8307bf690763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:16:16.582663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 624/1234] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:16:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:16:16.609701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1321e-cbef-4037-b042-11f3a91ae465 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:16.648187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:16.741768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1321e-cbef-4037-b042-11f3a91ae465 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:16.805076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af718a1-0253-420f-8f37-32ab188a21de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:16.805911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af718a1-0253-420f-8f37-32ab188a21de None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:16.806193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af718a1-0253-420f-8f37-32ab188a21de None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:16.806486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af718a1-0253-420f-8f37-32ab188a21de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:16.822710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af718a1-0253-420f-8f37-32ab188a21de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-9af718a1-0253-420f-8f37-32ab188a21de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:16:16.835187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:16.837108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:16.839044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 611/1235] 10.222.0.93 () {56 vars in 1102 bytes} [Fri Jun 3 15:16:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:16:16.844076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734ebc4-1be4-4ff9-a5d8-8307bf690763 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:16.911718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:16.982749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734ebc4-1be4-4ff9-a5d8-8307bf690763 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:17.035984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36464e15-5ee5-4736-9d89-8f6c18f79556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:17.036964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36464e15-5ee5-4736-9d89-8f6c18f79556 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:17.037425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36464e15-5ee5-4736-9d89-8f6c18f79556 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:17.037827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36464e15-5ee5-4736-9d89-8f6c18f79556 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:17.038921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36464e15-5ee5-4736-9d89-8f6c18f79556 None admin] RBAC: Adding query filter params (name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:17.039328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36464e15-5ee5-4736-9d89-8f6c18f79556 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:17.040885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36464e15-5ee5-4736-9d89-8f6c18f79556 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:17.050844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 625/1236] 10.222.0.93 () {56 vars in 1103 bytes} [Fri Jun 3 15:16:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:17.061496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af718a1-0253-420f-8f37-32ab188a21de None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:17.215937 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:17.238091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af718a1-0253-420f-8f37-32ab188a21de None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:17.284210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9542386-b173-4521-bfe5-7896d51decd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:17.285067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9542386-b173-4521-bfe5-7896d51decd3 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:17.285358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9542386-b173-4521-bfe5-7896d51decd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:17.285725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9542386-b173-4521-bfe5-7896d51decd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:17.298550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9542386-b173-4521-bfe5-7896d51decd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:17.300116 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9542386-b173-4521-bfe5-7896d51decd3 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-b9542386-b173-4521-bfe5-7896d51decd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:17.319602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:17.320656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:17.321679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:17.321679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:17.321679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:17.321679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:17.326598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 612/1237] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:17.327186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36464e15-5ee5-4736-9d89-8f6c18f79556 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:17.496892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36464e15-5ee5-4736-9d89-8f6c18f79556 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:17.511865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:17.548850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251fe16-c798-43ae-baa3-a43cb3db125c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:17.549440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251fe16-c798-43ae-baa3-a43cb3db125c None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:17.549723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251fe16-c798-43ae-baa3-a43cb3db125c None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:17.549993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2251fe16-c798-43ae-baa3-a43cb3db125c None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:17.550735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251fe16-c798-43ae-baa3-a43cb3db125c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:17.551036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251fe16-c798-43ae-baa3-a43cb3db125c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:17.552413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2251fe16-c798-43ae-baa3-a43cb3db125c None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:17.570926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 626/1238] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:17.578391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9542386-b173-4521-bfe5-7896d51decd3 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:17.740525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9542386-b173-4521-bfe5-7896d51decd3 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:17.786747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a5dea-8e7c-4d8b-8652-dd4c7c6cf034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:17.787451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a5dea-8e7c-4d8b-8652-dd4c7c6cf034 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:17.787950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a5dea-8e7c-4d8b-8652-dd4c7c6cf034 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:17.788264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a5dea-8e7c-4d8b-8652-dd4c7c6cf034 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:17.792788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:17.802369 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a5dea-8e7c-4d8b-8652-dd4c7c6cf034 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:17.804953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a5dea-8e7c-4d8b-8652-dd4c7c6cf034 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-ef8a5dea-8e7c-4d8b-8652-dd4c7c6cf034 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:16:17.831501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:17.833591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:17.835626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:17.835626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:17.835626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:16:17.835626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:16:17.835626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:16:17.835626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:16:17.835626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:16:17.835626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:16:17.835626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:17.835626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:17.835626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 613/1239] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:16:17.842569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251fe16-c798-43ae-baa3-a43cb3db125c None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:17.993962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2251fe16-c798-43ae-baa3-a43cb3db125c None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:18.040915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31653fdf-0b0c-41eb-9ba8-5cc2451fbd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:18.041694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31653fdf-0b0c-41eb-9ba8-5cc2451fbd55 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:18.042017 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31653fdf-0b0c-41eb-9ba8-5cc2451fbd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:18.042330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31653fdf-0b0c-41eb-9ba8-5cc2451fbd55 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:18.043363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31653fdf-0b0c-41eb-9ba8-5cc2451fbd55 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:18.043917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31653fdf-0b0c-41eb-9ba8-5cc2451fbd55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:18.045752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31653fdf-0b0c-41eb-9ba8-5cc2451fbd55 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:18.077284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 627/1240] 10.222.0.93 () {56 vars in 1143 bytes} [Fri Jun 3 15:16:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:18.080629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a5dea-8e7c-4d8b-8652-dd4c7c6cf034 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:18.236492 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a5dea-8e7c-4d8b-8652-dd4c7c6cf034 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:18.281179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6671d9b-4b5b-4fca-95cc-14c8ae8b7351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:18.282762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671d9b-4b5b-4fca-95cc-14c8ae8b7351 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:18.283144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671d9b-4b5b-4fca-95cc-14c8ae8b7351 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:18.283545 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6671d9b-4b5b-4fca-95cc-14c8ae8b7351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:18.374630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:18.385497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6671d9b-4b5b-4fca-95cc-14c8ae8b7351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:18.387560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6671d9b-4b5b-4fca-95cc-14c8ae8b7351 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:18.522382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 614/1241] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:16:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:16:18.529614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31653fdf-0b0c-41eb-9ba8-5cc2451fbd55 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:18.571807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:18.685444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31653fdf-0b0c-41eb-9ba8-5cc2451fbd55 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:18.731150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf94b9b-427d-4b64-b1d2-03a957edc631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:18.732050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf94b9b-427d-4b64-b1d2-03a957edc631 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:18.732317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf94b9b-427d-4b64-b1d2-03a957edc631 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:18.732575 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf94b9b-427d-4b64-b1d2-03a957edc631 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:18.746738 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf94b9b-427d-4b64-b1d2-03a957edc631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:18.748228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf94b9b-427d-4b64-b1d2-03a957edc631 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-bdf94b9b-427d-4b64-b1d2-03a957edc631 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:18.768687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:18.770553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:18.772367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:18.772367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:18.772367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:18.772367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:18.772367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 628/1242] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:18.777709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6671d9b-4b5b-4fca-95cc-14c8ae8b7351 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:18.906577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:18.915403 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6671d9b-4b5b-4fca-95cc-14c8ae8b7351 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:18.960023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cfcd2-28da-4176-a167-a3cbdb790507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:18.961231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cfcd2-28da-4176-a167-a3cbdb790507 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:18.961709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cfcd2-28da-4176-a167-a3cbdb790507 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:18.962279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cfcd2-28da-4176-a167-a3cbdb790507 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:18.963693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5cfcd2-28da-4176-a167-a3cbdb790507 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:18.964308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5cfcd2-28da-4176-a167-a3cbdb790507 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:18.966995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5cfcd2-28da-4176-a167-a3cbdb790507 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:18.990289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 615/1243] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:18.996736 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf94b9b-427d-4b64-b1d2-03a957edc631 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:19.112301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:19.143510 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf94b9b-427d-4b64-b1d2-03a957edc631 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:19.186761 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9829f-0c0f-4d22-b7c2-fbd6c4bd35b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:19.187706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9829f-0c0f-4d22-b7c2-fbd6c4bd35b4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:19.188607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9829f-0c0f-4d22-b7c2-fbd6c4bd35b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:19.188947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9829f-0c0f-4d22-b7c2-fbd6c4bd35b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:19.196443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9829f-0c0f-4d22-b7c2-fbd6c4bd35b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:19.197955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9829f-0c0f-4d22-b7c2-fbd6c4bd35b4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-57f9829f-0c0f-4d22-b7c2-fbd6c4bd35b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:16:19.217781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:19.219842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:19.221541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:16:19.221541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:16:19.221541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:16:19.221541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:16:19.221541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:16:19.221541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:16:19.221541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:19.221541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:19.221541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 629/1244] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:19.227636 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cfcd2-28da-4176-a167-a3cbdb790507 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:19.365386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cfcd2-28da-4176-a167-a3cbdb790507 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:19.412681 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9df48-209a-4d17-9285-0afa809344fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:19.413465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9df48-209a-4d17-9285-0afa809344fa None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:19.413772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9df48-209a-4d17-9285-0afa809344fa None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:19.414096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9df48-209a-4d17-9285-0afa809344fa None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:19.415024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9df48-209a-4d17-9285-0afa809344fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:19.415372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9df48-209a-4d17-9285-0afa809344fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:19.417034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9df48-209a-4d17-9285-0afa809344fa None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:19.434073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 616/1245] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:19.442087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9829f-0c0f-4d22-b7c2-fbd6c4bd35b4 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:19.590752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9829f-0c0f-4d22-b7c2-fbd6c4bd35b4 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:19.634725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b798c0a-ba80-4d30-ab51-234e3b9d5d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:19.635566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b798c0a-ba80-4d30-ab51-234e3b9d5d61 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:19.635986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b798c0a-ba80-4d30-ab51-234e3b9d5d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:19.636320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b798c0a-ba80-4d30-ab51-234e3b9d5d61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:19.637226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b798c0a-ba80-4d30-ab51-234e3b9d5d61 None admin] RBAC: Adding query filter params (role.id=cb4aa5570b15465eb4e81979b1019570, scope.project.id=23546323ca33412fb388afe308a97499, user.id=50625c2b38ed41698ba6cccdb12730ef) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:19.637564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b798c0a-ba80-4d30-ab51-234e3b9d5d61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:19.639187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b798c0a-ba80-4d30-ab51-234e3b9d5d61 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:19.645709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 630/1246] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:16:19 2022] GET /identity/v3/role_assignments?user.id=50625c2b38ed41698ba6cccdb12730ef&scope.project.id=23546323ca33412fb388afe308a97499&role.id=cb4aa5570b15465eb4e81979b1019570 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:22.159205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a201b-3ec8-4767-b035-f54acb6fb2ac None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:22.159926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a201b-3ec8-4767-b035-f54acb6fb2ac None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:22.159926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a201b-3ec8-4767-b035-f54acb6fb2ac None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:22.161497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 617/1247] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:16:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:16:22.168353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5de810-99c7-4df4-a0da-200d26eab9b2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:22.169336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5de810-99c7-4df4-a0da-200d26eab9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:22.169576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5de810-99c7-4df4-a0da-200d26eab9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:22.176154 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:22.193039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8b5de810-99c7-4df4-a0da-200d26eab9b2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:16:22.194638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8b5de810-99c7-4df4-a0da-200d26eab9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:16:22.286332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 631/1248] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:16:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:16:22.314300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a201b-3ec8-4767-b035-f54acb6fb2ac None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:22.337109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:22.441049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a201b-3ec8-4767-b035-f54acb6fb2ac None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:22.495904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7a51d-72e9-47dc-b478-84087429bef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:22.497262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7a51d-72e9-47dc-b478-84087429bef1 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:22.497782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7a51d-72e9-47dc-b478-84087429bef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:22.498264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7a51d-72e9-47dc-b478-84087429bef1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:22.513944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7a51d-72e9-47dc-b478-84087429bef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-a9a7a51d-72e9-47dc-b478-84087429bef1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:16:22.521648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:22.522690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:22.523595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 618/1249] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:16:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:16:22.530355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5de810-99c7-4df4-a0da-200d26eab9b2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:22.662231 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5de810-99c7-4df4-a0da-200d26eab9b2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:22.702809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:22.710913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e144fd-5d41-4b42-a6c3-fcb0bd5b8fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:22.711667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e144fd-5d41-4b42-a6c3-fcb0bd5b8fcb None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:22.711988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e144fd-5d41-4b42-a6c3-fcb0bd5b8fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:22.712303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e144fd-5d41-4b42-a6c3-fcb0bd5b8fcb None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:22.713281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e144fd-5d41-4b42-a6c3-fcb0bd5b8fcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:22.713724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e144fd-5d41-4b42-a6c3-fcb0bd5b8fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:22.715259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e144fd-5d41-4b42-a6c3-fcb0bd5b8fcb None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:22.727736 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 632/1250] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:16:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:22.734276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7a51d-72e9-47dc-b478-84087429bef1 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:22.833778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:22.864409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7a51d-72e9-47dc-b478-84087429bef1 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:22.908636 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177fba6-fedf-45f1-b427-ef11e69844b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:22.909920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177fba6-fedf-45f1-b427-ef11e69844b1 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:22.910440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177fba6-fedf-45f1-b427-ef11e69844b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:22.910955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177fba6-fedf-45f1-b427-ef11e69844b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:22.930165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177fba6-fedf-45f1-b427-ef11e69844b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:22.932918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177fba6-fedf-45f1-b427-ef11e69844b1 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-b177fba6-fedf-45f1-b427-ef11e69844b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:22.955814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:22.956890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:22.958969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:22.958969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:22.958969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:22.958969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:22.958969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 619/1251] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:16:22.963666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e144fd-5d41-4b42-a6c3-fcb0bd5b8fcb None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:23.110177 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e144fd-5d41-4b42-a6c3-fcb0bd5b8fcb None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:23.155770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23657ab-a58a-4fd6-a051-63c4d2989eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:23.157065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23657ab-a58a-4fd6-a051-63c4d2989eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:23.157348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23657ab-a58a-4fd6-a051-63c4d2989eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:23.157656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23657ab-a58a-4fd6-a051-63c4d2989eb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:23.158349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23657ab-a58a-4fd6-a051-63c4d2989eb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:23.158683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23657ab-a58a-4fd6-a051-63c4d2989eb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:23.160077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23657ab-a58a-4fd6-a051-63c4d2989eb1 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:23.178569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 633/1252] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:23.185742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177fba6-fedf-45f1-b427-ef11e69844b1 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:23.309381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177fba6-fedf-45f1-b427-ef11e69844b1 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:23.340755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:23.350935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67f2fd-5063-48a1-84cb-19f323f73667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:23.352091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67f2fd-5063-48a1-84cb-19f323f73667 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:23.352380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67f2fd-5063-48a1-84cb-19f323f73667 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:23.352660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67f2fd-5063-48a1-84cb-19f323f73667 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:23.362298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67f2fd-5063-48a1-84cb-19f323f73667 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:23.363776 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67f2fd-5063-48a1-84cb-19f323f73667 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-8a67f2fd-5063-48a1-84cb-19f323f73667 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:16:23.379980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:23.381250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:23.387356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:23.387356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:23.387356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:16:23.387356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:16:23.387356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:16:23.387356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:16:23.387356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:16:23.387356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:16:23.387356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:23.387356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:23.387356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 620/1253] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:16:23.388109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23657ab-a58a-4fd6-a051-63c4d2989eb1 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:23.467077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:23.522833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23657ab-a58a-4fd6-a051-63c4d2989eb1 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:23.564625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10f9a1-2638-49d5-9a82-099aadbebef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:23.565289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f9a1-2638-49d5-9a82-099aadbebef2 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:23.565576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f9a1-2638-49d5-9a82-099aadbebef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:23.565989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10f9a1-2638-49d5-9a82-099aadbebef2 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:23.566958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10f9a1-2638-49d5-9a82-099aadbebef2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:23.567224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10f9a1-2638-49d5-9a82-099aadbebef2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:23.568830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10f9a1-2638-49d5-9a82-099aadbebef2 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:23.593657 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 634/1254] 10.222.0.93 () {56 vars in 1143 bytes} [Fri Jun 3 15:16:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:23.600760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67f2fd-5063-48a1-84cb-19f323f73667 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:23.733552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67f2fd-5063-48a1-84cb-19f323f73667 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:23.777195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3488e64-5d54-44e7-afbc-2e5ef8845bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:23.777967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488e64-5d54-44e7-afbc-2e5ef8845bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:23.778244 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488e64-5d54-44e7-afbc-2e5ef8845bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:23.778522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3488e64-5d54-44e7-afbc-2e5ef8845bc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:23.850787 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:23.865376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3488e64-5d54-44e7-afbc-2e5ef8845bc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:23.866867 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3488e64-5d54-44e7-afbc-2e5ef8845bc3 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:23.992809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 621/1255] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:16:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 03 15:16:24.001446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10f9a1-2638-49d5-9a82-099aadbebef2 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:24.148073 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10f9a1-2638-49d5-9a82-099aadbebef2 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:24.165324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:24.194800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc94cb0-fe93-4ea2-89aa-2568647877f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:24.195853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc94cb0-fe93-4ea2-89aa-2568647877f0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:24.196251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc94cb0-fe93-4ea2-89aa-2568647877f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:24.196686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc94cb0-fe93-4ea2-89aa-2568647877f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:24.211123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc94cb0-fe93-4ea2-89aa-2568647877f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:24.213393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc94cb0-fe93-4ea2-89aa-2568647877f0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-abc94cb0-fe93-4ea2-89aa-2568647877f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:24.232512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:24.233728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:24.235488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:24.235488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:24.235488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:24.235488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:24.235488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 635/1256] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:24.240528 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3488e64-5d54-44e7-afbc-2e5ef8845bc3 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:24.370400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:24.385085 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3488e64-5d54-44e7-afbc-2e5ef8845bc3 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:24.439547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d45a5e-3471-41e5-b9da-bc95a16b864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:24.440273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d45a5e-3471-41e5-b9da-bc95a16b864a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:24.440590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d45a5e-3471-41e5-b9da-bc95a16b864a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:24.440886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d45a5e-3471-41e5-b9da-bc95a16b864a None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:24.441603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d45a5e-3471-41e5-b9da-bc95a16b864a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:24.441910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d45a5e-3471-41e5-b9da-bc95a16b864a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:24.443571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d45a5e-3471-41e5-b9da-bc95a16b864a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:24.457047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 622/1257] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:24.463298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc94cb0-fe93-4ea2-89aa-2568647877f0 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:24.609308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc94cb0-fe93-4ea2-89aa-2568647877f0 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:24.659115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd2bd9-66af-4ae0-a826-c51c6a8ae7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:24.660074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd2bd9-66af-4ae0-a826-c51c6a8ae7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:24.660414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd2bd9-66af-4ae0-a826-c51c6a8ae7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:24.660693 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd2bd9-66af-4ae0-a826-c51c6a8ae7c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:24.668990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd2bd9-66af-4ae0-a826-c51c6a8ae7c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:24.670760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd2bd9-66af-4ae0-a826-c51c6a8ae7c8 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-8cdd2bd9-66af-4ae0-a826-c51c6a8ae7c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:16:24.685237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:24.686285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:24.687921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:16:24.687921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:16:24.687921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:16:24.687921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:16:24.687921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:16:24.687921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:16:24.687921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:24.687921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:24.687921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 636/1258] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:24.692516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d45a5e-3471-41e5-b9da-bc95a16b864a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:24.812329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d45a5e-3471-41e5-b9da-bc95a16b864a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:24.849454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce918b-c28b-403a-a578-6807199619c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:24.850118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce918b-c28b-403a-a578-6807199619c6 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:24.850423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce918b-c28b-403a-a578-6807199619c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:24.850761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce918b-c28b-403a-a578-6807199619c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:24.851796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce918b-c28b-403a-a578-6807199619c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:24.852316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce918b-c28b-403a-a578-6807199619c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:24.854493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce918b-c28b-403a-a578-6807199619c6 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:24.868379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 623/1259] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:24.874488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd2bd9-66af-4ae0-a826-c51c6a8ae7c8 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:24.910587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:25.011624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd2bd9-66af-4ae0-a826-c51c6a8ae7c8 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:25.067690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85d425-02f3-4db4-a4e9-bca961fe7588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:25.068424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85d425-02f3-4db4-a4e9-bca961fe7588 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:25.068711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85d425-02f3-4db4-a4e9-bca961fe7588 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:25.069028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85d425-02f3-4db4-a4e9-bca961fe7588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:25.070260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85d425-02f3-4db4-a4e9-bca961fe7588 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=23546323ca33412fb388afe308a97499, user.id=50625c2b38ed41698ba6cccdb12730ef) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:25.070807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85d425-02f3-4db4-a4e9-bca961fe7588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:25.072962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85d425-02f3-4db4-a4e9-bca961fe7588 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:25.081597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 637/1260] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:16:24 2022] GET /identity/v3/role_assignments?user.id=50625c2b38ed41698ba6cccdb12730ef&scope.project.id=23546323ca33412fb388afe308a97499&role.id=d67707782358417ba9f25b2793ca2890 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:27.451064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ce8f6-68a7-4816-a3e4-f99afd3c1a07 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:27.451603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ce8f6-68a7-4816-a3e4-f99afd3c1a07 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:27.451603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ce8f6-68a7-4816-a3e4-f99afd3c1a07 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:27.452985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 624/1261] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:16:27.460112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe826ef-3179-438a-87bd-b39cfffb185a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:27.460550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe826ef-3179-438a-87bd-b39cfffb185a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:27.460672 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe826ef-3179-438a-87bd-b39cfffb185a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:27.480331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3fe826ef-3179-438a-87bd-b39cfffb185a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:16:27.481508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3fe826ef-3179-438a-87bd-b39cfffb185a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:16:27.555188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 638/1262] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:16:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:16:27.577372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ce8f6-68a7-4816-a3e4-f99afd3c1a07 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:27.588488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:27.689973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ce8f6-68a7-4816-a3e4-f99afd3c1a07 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:27.733391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46f72e-d76c-4be6-813f-490441cc5075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:27.734678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46f72e-d76c-4be6-813f-490441cc5075 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:27.735163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46f72e-d76c-4be6-813f-490441cc5075 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:27.735645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46f72e-d76c-4be6-813f-490441cc5075 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:27.748966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46f72e-d76c-4be6-813f-490441cc5075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-5d46f72e-d76c-4be6-813f-490441cc5075 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:16:27.756619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:27.757846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:27.758763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 625/1263] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:16:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:27.767377 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe826ef-3179-438a-87bd-b39cfffb185a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:27.788919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:27.889105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe826ef-3179-438a-87bd-b39cfffb185a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:27.932637 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936f87-ae8a-4e8c-a989-e438db680687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:27.933305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936f87-ae8a-4e8c-a989-e438db680687 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:27.933590 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936f87-ae8a-4e8c-a989-e438db680687 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:27.933863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e936f87-ae8a-4e8c-a989-e438db680687 None admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:27.934633 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936f87-ae8a-4e8c-a989-e438db680687 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:27.934944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936f87-ae8a-4e8c-a989-e438db680687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:27.936378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e936f87-ae8a-4e8c-a989-e438db680687 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:27.944753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 639/1264] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:16:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:27.950828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46f72e-d76c-4be6-813f-490441cc5075 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:28.023868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:28.068505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46f72e-d76c-4be6-813f-490441cc5075 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:28.115289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176903-f216-4744-b9a0-7eef3a75e64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:28.116082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176903-f216-4744-b9a0-7eef3a75e64c None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:28.116372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176903-f216-4744-b9a0-7eef3a75e64c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:28.116649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176903-f216-4744-b9a0-7eef3a75e64c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:28.134764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90176903-f216-4744-b9a0-7eef3a75e64c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:28.136243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90176903-f216-4744-b9a0-7eef3a75e64c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-90176903-f216-4744-b9a0-7eef3a75e64c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:28.155158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:28.156765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:28.157952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:28.157952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:28.157952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:28.157952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:28.157952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 626/1265] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:28.164501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936f87-ae8a-4e8c-a989-e438db680687 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:28.259235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:28.280304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e936f87-ae8a-4e8c-a989-e438db680687 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:28.317836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99725c02-a6d9-466d-9615-aa7b37e3af61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:28.318741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99725c02-a6d9-466d-9615-aa7b37e3af61 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:28.319171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99725c02-a6d9-466d-9615-aa7b37e3af61 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:28.319578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99725c02-a6d9-466d-9615-aa7b37e3af61 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:28.320408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99725c02-a6d9-466d-9615-aa7b37e3af61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:28.320835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99725c02-a6d9-466d-9615-aa7b37e3af61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:28.322358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99725c02-a6d9-466d-9615-aa7b37e3af61 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:28.338214 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 640/1266] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:28.344512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176903-f216-4744-b9a0-7eef3a75e64c None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:28.463145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176903-f216-4744-b9a0-7eef3a75e64c None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:28.486372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:28.509416 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99693f-bff4-4f3e-8ab0-569ffcadd812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:28.510235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99693f-bff4-4f3e-8ab0-569ffcadd812 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:28.510522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99693f-bff4-4f3e-8ab0-569ffcadd812 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:28.510830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99693f-bff4-4f3e-8ab0-569ffcadd812 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:28.520974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99693f-bff4-4f3e-8ab0-569ffcadd812 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:28.522462 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99693f-bff4-4f3e-8ab0-569ffcadd812 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-3b99693f-bff4-4f3e-8ab0-569ffcadd812 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:16:28.539549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:28.540751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:28.541917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:28.541917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:28.541917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:16:28.541917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:16:28.541917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:16:28.541917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:16:28.541917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:16:28.541917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:16:28.541917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:28.541917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:28.541917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 627/1267] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:28.546892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99725c02-a6d9-466d-9615-aa7b37e3af61 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:28.669798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99725c02-a6d9-466d-9615-aa7b37e3af61 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:28.722252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee8015-8e0e-4eeb-a36c-2f72ed34a625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:28.722983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee8015-8e0e-4eeb-a36c-2f72ed34a625 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:28.723275 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee8015-8e0e-4eeb-a36c-2f72ed34a625 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:28.723549 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee8015-8e0e-4eeb-a36c-2f72ed34a625 None admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:28.724405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee8015-8e0e-4eeb-a36c-2f72ed34a625 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:28.724712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee8015-8e0e-4eeb-a36c-2f72ed34a625 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:28.726195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee8015-8e0e-4eeb-a36c-2f72ed34a625 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:28.736325 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:28.746157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 641/1268] 10.222.0.93 () {56 vars in 1143 bytes} [Fri Jun 3 15:16:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:28.753142 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99693f-bff4-4f3e-8ab0-569ffcadd812 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:28.883343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99693f-bff4-4f3e-8ab0-569ffcadd812 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:28.920491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24423b3c-639c-415e-8474-8c105606fa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:28.921307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24423b3c-639c-415e-8474-8c105606fa66 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:28.921614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24423b3c-639c-415e-8474-8c105606fa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:28.921888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24423b3c-639c-415e-8474-8c105606fa66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:28.933823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24423b3c-639c-415e-8474-8c105606fa66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:28.935338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24423b3c-639c-415e-8474-8c105606fa66 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-24423b3c-639c-415e-8474-8c105606fa66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:28.955555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:28.957288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:28.958506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:28.958506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:28.958506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:28.958506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:28.958506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 628/1269] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 03 15:16:28.964394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee8015-8e0e-4eeb-a36c-2f72ed34a625 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:29.121302 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee8015-8e0e-4eeb-a36c-2f72ed34a625 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:29.173016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ed7b6-d6d7-48b3-9484-93ae57093ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:29.174373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ed7b6-d6d7-48b3-9484-93ae57093ccc None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:29.174900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ed7b6-d6d7-48b3-9484-93ae57093ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:29.175731 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ed7b6-d6d7-48b3-9484-93ae57093ccc None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:29.177462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ed7b6-d6d7-48b3-9484-93ae57093ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:29.178015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ed7b6-d6d7-48b3-9484-93ae57093ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:29.180955 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ed7b6-d6d7-48b3-9484-93ae57093ccc None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:29.200337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 642/1270] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:29.206428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24423b3c-639c-415e-8474-8c105606fa66 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:29.242446 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:29.320558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24423b3c-639c-415e-8474-8c105606fa66 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:29.377284 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fdcac-3c0c-46b1-b23d-2272d022ef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:29.378078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fdcac-3c0c-46b1-b23d-2272d022ef0e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:29.378362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fdcac-3c0c-46b1-b23d-2272d022ef0e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:29.378675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fdcac-3c0c-46b1-b23d-2272d022ef0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:29.385705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fdcac-3c0c-46b1-b23d-2272d022ef0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:29.387261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fdcac-3c0c-46b1-b23d-2272d022ef0e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-295fdcac-3c0c-46b1-b23d-2272d022ef0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:16:29.401767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:29.403118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:29.404024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:16:29.404024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:16:29.404024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:16:29.404024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:16:29.404024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:16:29.404024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:16:29.404024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:29.404024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:29.404024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 629/1271] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:29.409450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ed7b6-d6d7-48b3-9484-93ae57093ccc None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:29.478859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:29.536012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ed7b6-d6d7-48b3-9484-93ae57093ccc None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:29.577751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1cc3e-c41c-429d-bbe2-d457807495a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:29.579566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1cc3e-c41c-429d-bbe2-d457807495a0 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:29.579566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1cc3e-c41c-429d-bbe2-d457807495a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:29.579566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1cc3e-c41c-429d-bbe2-d457807495a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:29.579840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1cc3e-c41c-429d-bbe2-d457807495a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:29.580054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1cc3e-c41c-429d-bbe2-d457807495a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:29.581495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1cc3e-c41c-429d-bbe2-d457807495a0 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:29.596014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 643/1272] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:29.602568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fdcac-3c0c-46b1-b23d-2272d022ef0e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:29.731938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:29.743200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fdcac-3c0c-46b1-b23d-2272d022ef0e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:29.779417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed543159-da2d-41f7-b677-3742119d1b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:29.780550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed543159-da2d-41f7-b677-3742119d1b5c None admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:29.780833 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed543159-da2d-41f7-b677-3742119d1b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:29.781654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed543159-da2d-41f7-b677-3742119d1b5c None admin] PATH_INFO: `/v3/projects/23546323ca33412fb388afe308a97499/users/50625c2b38ed41698ba6cccdb12730ef/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:29.783651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed543159-da2d-41f7-b677-3742119d1b5c None admin] RBAC: Authorizing `identity:create_grant(project_id=23546323ca33412fb388afe308a97499, user_id=50625c2b38ed41698ba6cccdb12730ef, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:29.785102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed543159-da2d-41f7-b677-3742119d1b5c None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:29.799454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 630/1273] 10.222.0.93 () {58 vars in 1439 bytes} [Fri Jun 3 15:16:29 2022] PUT /identity/v3/projects/23546323ca33412fb388afe308a97499/users/50625c2b38ed41698ba6cccdb12730ef/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:16:32.135136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c5e1a-98b6-43c9-87bb-8484fb24851b None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:32.135835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c5e1a-98b6-43c9-87bb-8484fb24851b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:32.135835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c5e1a-98b6-43c9-87bb-8484fb24851b None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:32.137069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 644/1274] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:16:32.144437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf824e-0776-4896-ab64-c3026dc3caf2 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:32.144997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf824e-0776-4896-ab64-c3026dc3caf2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:32.145093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf824e-0776-4896-ab64-c3026dc3caf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:32.164941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-fecf824e-0776-4896-ab64-c3026dc3caf2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:16:32.166144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-fecf824e-0776-4896-ab64-c3026dc3caf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:16:32.234992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 631/1275] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:16:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:16:32.258873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c5e1a-98b6-43c9-87bb-8484fb24851b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:32.369015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c5e1a-98b6-43c9-87bb-8484fb24851b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:32.397470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:32.416483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a06d0-bea3-4bbb-b826-751944f013f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:32.417273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a06d0-bea3-4bbb-b826-751944f013f5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:32.417560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a06d0-bea3-4bbb-b826-751944f013f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:32.417838 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a06d0-bea3-4bbb-b826-751944f013f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:32.428153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a06d0-bea3-4bbb-b826-751944f013f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-cf8a06d0-bea3-4bbb-b826-751944f013f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 03 15:16:32.435773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:32.437518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:32.439297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 645/1276] 10.222.0.93 () {56 vars in 1096 bytes} [Fri Jun 3 15:16:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:32.443717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf824e-0776-4896-ab64-c3026dc3caf2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:32.538161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:32.557797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf824e-0776-4896-ab64-c3026dc3caf2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:32.595780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01229f-9c7c-4e8c-b0fb-9331191bf6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:32.596448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01229f-9c7c-4e8c-b0fb-9331191bf6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:32.596734 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01229f-9c7c-4e8c-b0fb-9331191bf6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:32.596990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01229f-9c7c-4e8c-b0fb-9331191bf6f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:32.597743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01229f-9c7c-4e8c-b0fb-9331191bf6f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:32.598049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01229f-9c7c-4e8c-b0fb-9331191bf6f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:32.599469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01229f-9c7c-4e8c-b0fb-9331191bf6f8 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:32.608238 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 632/1277] 10.222.0.93 () {56 vars in 1099 bytes} [Fri Jun 3 15:16:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:32.614496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a06d0-bea3-4bbb-b826-751944f013f5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:32.725692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a06d0-bea3-4bbb-b826-751944f013f5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:32.763197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884338ea-4355-4abb-b8eb-99ea9580e24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:32.763984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884338ea-4355-4abb-b8eb-99ea9580e24d None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:32.764263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884338ea-4355-4abb-b8eb-99ea9580e24d None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:32.764536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884338ea-4355-4abb-b8eb-99ea9580e24d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:32.775723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884338ea-4355-4abb-b8eb-99ea9580e24d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:32.777286 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884338ea-4355-4abb-b8eb-99ea9580e24d None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-884338ea-4355-4abb-b8eb-99ea9580e24d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:32.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:32.804530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:32.805696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:32.805696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:32.805696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:32.805696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:32.805696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 646/1278] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:32.814318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01229f-9c7c-4e8c-b0fb-9331191bf6f8 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:32.923362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01229f-9c7c-4e8c-b0fb-9331191bf6f8 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:32.962806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded08c8d-66f3-4046-8676-8d2ccd3a4af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:32.963384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded08c8d-66f3-4046-8676-8d2ccd3a4af2 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:32.963662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded08c8d-66f3-4046-8676-8d2ccd3a4af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:32.963939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded08c8d-66f3-4046-8676-8d2ccd3a4af2 None admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:32.964668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded08c8d-66f3-4046-8676-8d2ccd3a4af2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:32.964970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded08c8d-66f3-4046-8676-8d2ccd3a4af2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:32.966351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded08c8d-66f3-4046-8676-8d2ccd3a4af2 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:32.979215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 633/1279] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:32.985954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884338ea-4355-4abb-b8eb-99ea9580e24d None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:33.023305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:33.106641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884338ea-4355-4abb-b8eb-99ea9580e24d None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:33.160484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc41e01-ca36-476e-b3f0-692636773d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:33.161447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc41e01-ca36-476e-b3f0-692636773d53 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:33.161642 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc41e01-ca36-476e-b3f0-692636773d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:33.161818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc41e01-ca36-476e-b3f0-692636773d53 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:33.172636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc41e01-ca36-476e-b3f0-692636773d53 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:33.175419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc41e01-ca36-476e-b3f0-692636773d53 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-4dc41e01-ca36-476e-b3f0-692636773d53 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 03 15:16:33.198965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:33.200769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:33.202705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:33.202705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:33.202705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 03 15:16:33.202705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 03 15:16:33.202705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 03 15:16:33.202705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 03 15:16:33.202705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 03 15:16:33.202705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 03 15:16:33.202705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 03 15:16:33.202705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:33.202705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 647/1280] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:33.209019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded08c8d-66f3-4046-8676-8d2ccd3a4af2 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:33.214698 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:33.318133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded08c8d-66f3-4046-8676-8d2ccd3a4af2 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:33.363437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1091dd-c722-4146-90b1-ff36e2a5264e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:33.364139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1091dd-c722-4146-90b1-ff36e2a5264e None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:33.364480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1091dd-c722-4146-90b1-ff36e2a5264e None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:33.364812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1091dd-c722-4146-90b1-ff36e2a5264e None admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:33.365767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1091dd-c722-4146-90b1-ff36e2a5264e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:33.366131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1091dd-c722-4146-90b1-ff36e2a5264e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:33.367840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1091dd-c722-4146-90b1-ff36e2a5264e None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:33.390808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 634/1281] 10.222.0.93 () {56 vars in 1143 bytes} [Fri Jun 3 15:16:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:33.397284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc41e01-ca36-476e-b3f0-692636773d53 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:33.508755 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:33.519091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc41e01-ca36-476e-b3f0-692636773d53 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:33.555180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee39fe-f5d6-4cc0-be21-3a6f3c71f14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:33.555881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee39fe-f5d6-4cc0-be21-3a6f3c71f14a None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:33.556099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee39fe-f5d6-4cc0-be21-3a6f3c71f14a None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:33.556369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee39fe-f5d6-4cc0-be21-3a6f3c71f14a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:33.625087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee39fe-f5d6-4cc0-be21-3a6f3c71f14a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:33.626558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee39fe-f5d6-4cc0-be21-3a6f3c71f14a None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:33.720347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:33.731739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 648/1282] 10.222.0.93 () {58 vars in 1282 bytes} [Fri Jun 3 15:16:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 03 15:16:33.739039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1091dd-c722-4146-90b1-ff36e2a5264e None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:33.804411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:33.846647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1091dd-c722-4146-90b1-ff36e2a5264e None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:33.881792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5e6aa-9a3c-420c-a63f-a4877839dd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:33.882610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5e6aa-9a3c-420c-a63f-a4877839dd51 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:33.882893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5e6aa-9a3c-420c-a63f-a4877839dd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:33.883172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5e6aa-9a3c-420c-a63f-a4877839dd51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:33.894126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5e6aa-9a3c-420c-a63f-a4877839dd51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:33.895619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d5e6aa-9a3c-420c-a63f-a4877839dd51 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-42d5e6aa-9a3c-420c-a63f-a4877839dd51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 03 15:16:33.914037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:33.915310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:33.916393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 03 15:16:33.916393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 03 15:16:33.916393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 03 15:16:33.916393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:33.916393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 635/1283] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:33.922230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee39fe-f5d6-4cc0-be21-3a6f3c71f14a None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:34.025645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee39fe-f5d6-4cc0-be21-3a6f3c71f14a None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:34.068337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c838b-1efe-41e3-bff6-058665cdb722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:34.068909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c838b-1efe-41e3-bff6-058665cdb722 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:34.069186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c838b-1efe-41e3-bff6-058665cdb722 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:34.069479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c838b-1efe-41e3-bff6-058665cdb722 None admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:34.070218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c838b-1efe-41e3-bff6-058665cdb722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:34.070619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c838b-1efe-41e3-bff6-058665cdb722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:34.072013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c838b-1efe-41e3-bff6-058665cdb722 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:34.086041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 649/1284] 10.222.0.93 () {56 vars in 1109 bytes} [Fri Jun 3 15:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:34.092132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5e6aa-9a3c-420c-a63f-a4877839dd51 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:34.212409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d5e6aa-9a3c-420c-a63f-a4877839dd51 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:34.230868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:34.268979 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b1952-ae00-4fe2-9de0-5fa163b37c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:34.270967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b1952-ae00-4fe2-9de0-5fa163b37c99 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:34.271582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b1952-ae00-4fe2-9de0-5fa163b37c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:34.272330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33b1952-ae00-4fe2-9de0-5fa163b37c99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:34.282393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b1952-ae00-4fe2-9de0-5fa163b37c99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:34.283892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33b1952-ae00-4fe2-9de0-5fa163b37c99 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application [None req-f33b1952-ae00-4fe2-9de0-5fa163b37c99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 03 15:16:34.299743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:34.301057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:34.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 03 15:16:34.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 03 15:16:34.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 03 15:16:34.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 03 15:16:34.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 03 15:16:34.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 03 15:16:34.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 03 15:16:34.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: ERROR keystone.server.flask.application Jun 03 15:16:34.302252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 636/1285] 10.222.0.93 () {56 vars in 1111 bytes} [Fri Jun 3 15:16:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:34.308986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c838b-1efe-41e3-bff6-058665cdb722 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:34.365240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:34.416626 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c838b-1efe-41e3-bff6-058665cdb722 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:34.453242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb4b28-3f75-4778-a390-b12353736b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:34.453902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb4b28-3f75-4778-a390-b12353736b74 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:34.454183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb4b28-3f75-4778-a390-b12353736b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:34.454462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb4b28-3f75-4778-a390-b12353736b74 None admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:34.455289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb4b28-3f75-4778-a390-b12353736b74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:34.455603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb4b28-3f75-4778-a390-b12353736b74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:34.457056 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb4b28-3f75-4778-a390-b12353736b74 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:34.469878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 650/1286] 10.222.0.93 () {56 vars in 1148 bytes} [Fri Jun 3 15:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:34.476206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b1952-ae00-4fe2-9de0-5fa163b37c99 None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:34.585513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33b1952-ae00-4fe2-9de0-5fa163b37c99 None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:34.620693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57feba91-7b05-461f-a406-3de509d93b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:34.621256 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57feba91-7b05-461f-a406-3de509d93b48 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:34.621540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57feba91-7b05-461f-a406-3de509d93b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:34.621818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57feba91-7b05-461f-a406-3de509d93b48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:34.622689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57feba91-7b05-461f-a406-3de509d93b48 None admin] RBAC: Adding query filter params (role.id=d67707782358417ba9f25b2793ca2890, scope.project.id=23546323ca33412fb388afe308a97499, user.id=50625c2b38ed41698ba6cccdb12730ef) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:34.623003 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57feba91-7b05-461f-a406-3de509d93b48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:34.624516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57feba91-7b05-461f-a406-3de509d93b48 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:34.631512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 637/1287] 10.222.0.93 () {56 vars in 1374 bytes} [Fri Jun 3 15:16:34 2022] GET /identity/v3/role_assignments?user.id=50625c2b38ed41698ba6cccdb12730ef&scope.project.id=23546323ca33412fb388afe308a97499&role.id=d67707782358417ba9f25b2793ca2890 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:37.014458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b30b85-e69b-4232-b548-ea250818de42 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:37.015563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b30b85-e69b-4232-b548-ea250818de42 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:37.015699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b30b85-e69b-4232-b548-ea250818de42 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:37.017150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 651/1288] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:16:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:16:37.023237 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f22eec-2a1d-48d1-b9dd-1dc4d11aa318 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:37.023736 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f22eec-2a1d-48d1-b9dd-1dc4d11aa318 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:37.023931 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f22eec-2a1d-48d1-b9dd-1dc4d11aa318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:37.042246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-60f22eec-2a1d-48d1-b9dd-1dc4d11aa318 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:16:37.043474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-60f22eec-2a1d-48d1-b9dd-1dc4d11aa318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:16:37.083303 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:37.125909 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 638/1289] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:16:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:16:37.149091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b30b85-e69b-4232-b548-ea250818de42 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:37.283419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b30b85-e69b-4232-b548-ea250818de42 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:37.300619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:37.329182 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fb4a5-c762-45c7-96d7-e8c4004243f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:37.329979 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fb4a5-c762-45c7-96d7-e8c4004243f5 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:37.330261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fb4a5-c762-45c7-96d7-e8c4004243f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:37.330558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fb4a5-c762-45c7-96d7-e8c4004243f5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:37.331227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fb4a5-c762-45c7-96d7-e8c4004243f5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:37.332611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fb4a5-c762-45c7-96d7-e8c4004243f5 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-561fb4a5-c762-45c7-96d7-e8c4004243f5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:37.345079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:37.346737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:37.347846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 652/1290] 10.222.0.93 () {56 vars in 1117 bytes} [Fri Jun 3 15:16:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:37.353744 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f22eec-2a1d-48d1-b9dd-1dc4d11aa318 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:37.465097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f22eec-2a1d-48d1-b9dd-1dc4d11aa318 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:37.503682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd29b11-8cff-41e3-9cf2-c1485f6e8eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:37.504411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29b11-8cff-41e3-9cf2-c1485f6e8eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:37.504695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29b11-8cff-41e3-9cf2-c1485f6e8eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:37.504963 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29b11-8cff-41e3-9cf2-c1485f6e8eb9 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:37.505774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd29b11-8cff-41e3-9cf2-c1485f6e8eb9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:37.506077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd29b11-8cff-41e3-9cf2-c1485f6e8eb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:37.507702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd29b11-8cff-41e3-9cf2-c1485f6e8eb9 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:37.516823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 639/1291] 10.222.0.93 () {56 vars in 1116 bytes} [Fri Jun 3 15:16:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:37.523130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fb4a5-c762-45c7-96d7-e8c4004243f5 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:37.637845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fb4a5-c762-45c7-96d7-e8c4004243f5 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:37.680037 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c321f3-d6c1-46b6-9732-4ac7393000c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:37.681144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c321f3-d6c1-46b6-9732-4ac7393000c4 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:37.681641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c321f3-d6c1-46b6-9732-4ac7393000c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:37.682083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c321f3-d6c1-46b6-9732-4ac7393000c4 None admin] PATH_INFO: `/v3/services` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:37.683104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c321f3-d6c1-46b6-9732-4ac7393000c4 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:37.683659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c321f3-d6c1-46b6-9732-4ac7393000c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:37.685548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c321f3-d6c1-46b6-9732-4ac7393000c4 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:37.690994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 653/1292] 10.222.0.93 () {56 vars in 1116 bytes} [Fri Jun 3 15:16:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:40.031270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0845-f40a-4e55-81fe-235e6817466f None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:40.031991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0845-f40a-4e55-81fe-235e6817466f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:40.031991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0845-f40a-4e55-81fe-235e6817466f None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:40.033234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 640/1293] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:16:40.043399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f042407-3bc6-481b-accb-e55dc7591f0e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:40.044234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f042407-3bc6-481b-accb-e55dc7591f0e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:40.044234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f042407-3bc6-481b-accb-e55dc7591f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:40.063447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3f042407-3bc6-481b-accb-e55dc7591f0e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:16:40.064797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3f042407-3bc6-481b-accb-e55dc7591f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:16:40.115195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:40.171699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 654/1294] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:16:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:16:40.195601 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d0845-f40a-4e55-81fe-235e6817466f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:40.239712 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:40.317662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d0845-f40a-4e55-81fe-235e6817466f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:40.353404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e26283-c12e-4e66-ba15-982fba645259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:40.354068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26283-c12e-4e66-ba15-982fba645259 None admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:40.354348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26283-c12e-4e66-ba15-982fba645259 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:40.354651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e26283-c12e-4e66-ba15-982fba645259 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:40.355398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e26283-c12e-4e66-ba15-982fba645259 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:40.356851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e26283-c12e-4e66-ba15-982fba645259 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:40.372752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 641/1295] 10.222.0.93 () {60 vars in 1141 bytes} [Fri Jun 3 15:16:40 2022] POST /identity/v3/services => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:16:42.686984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da606bc-5da4-44d7-9953-bfdbba19a863 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:42.687570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da606bc-5da4-44d7-9953-bfdbba19a863 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:42.687823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da606bc-5da4-44d7-9953-bfdbba19a863 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:42.690247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 655/1296] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:16:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:16:42.698206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699a0f5-70d1-40c0-9238-60d909d805ed None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:42.698780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699a0f5-70d1-40c0-9238-60d909d805ed None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:42.698780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699a0f5-70d1-40c0-9238-60d909d805ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:42.718809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-6699a0f5-70d1-40c0-9238-60d909d805ed None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:16:42.720104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6699a0f5-70d1-40c0-9238-60d909d805ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:16:42.820667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 642/1297] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:16:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 03 15:16:42.844964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da606bc-5da4-44d7-9953-bfdbba19a863 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:42.966515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da606bc-5da4-44d7-9953-bfdbba19a863 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:43.006996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e25de-6e1a-4702-b860-02f132e501fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:43.007807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e25de-6e1a-4702-b860-02f132e501fe None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:43.008087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e25de-6e1a-4702-b860-02f132e501fe None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:43.008360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e25de-6e1a-4702-b860-02f132e501fe None admin] PATH_INFO: `/v3/services/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:43.009016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e25de-6e1a-4702-b860-02f132e501fe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:43.010398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e25de-6e1a-4702-b860-02f132e501fe None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-635e25de-6e1a-4702-b860-02f132e501fe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:43.023759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:43.025524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:43.026775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 656/1298] 10.222.0.93 () {56 vars in 1117 bytes} [Fri Jun 3 15:16:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:43.031806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699a0f5-70d1-40c0-9238-60d909d805ed None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:43.058626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:43.150032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699a0f5-70d1-40c0-9238-60d909d805ed None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:43.191102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62635ed-7bf0-450b-a9ce-c8739b5c009a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:43.191774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62635ed-7bf0-450b-a9ce-c8739b5c009a None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:43.192056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62635ed-7bf0-450b-a9ce-c8739b5c009a None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:43.192332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62635ed-7bf0-450b-a9ce-c8739b5c009a None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:43.193044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62635ed-7bf0-450b-a9ce-c8739b5c009a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:43.193353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62635ed-7bf0-450b-a9ce-c8739b5c009a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:43.194878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62635ed-7bf0-450b-a9ce-c8739b5c009a None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:43.202727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 643/1299] 10.222.0.93 () {56 vars in 1116 bytes} [Fri Jun 3 15:16:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:16:43.209239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e25de-6e1a-4702-b860-02f132e501fe None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:43.248833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:43.324666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e25de-6e1a-4702-b860-02f132e501fe None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:43.364357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d8283-0dc8-4481-90d4-9c801dd45a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:43.365102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d8283-0dc8-4481-90d4-9c801dd45a7f None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:43.365433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d8283-0dc8-4481-90d4-9c801dd45a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:43.365758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d8283-0dc8-4481-90d4-9c801dd45a7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:43.366675 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d8283-0dc8-4481-90d4-9c801dd45a7f None admin] RBAC: Adding query filter params (interface=public, service_id=03ed438ca134441989055461a5f222fc, region_id=RegionOne) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:43.367140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d8283-0dc8-4481-90d4-9c801dd45a7f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:43.368650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d8283-0dc8-4481-90d4-9c801dd45a7f None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:43.375072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 657/1300] 10.222.0.93 () {56 vars in 1251 bytes} [Fri Jun 3 15:16:43 2022] GET /identity/v3/endpoints?service_id=03ed438ca134441989055461a5f222fc&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:43.382625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62635ed-7bf0-450b-a9ce-c8739b5c009a None admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:43.509170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:43.532445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62635ed-7bf0-450b-a9ce-c8739b5c009a None admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:43.579241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8dc4da-9218-4cd6-9065-528ef49d4a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:43.579913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dc4da-9218-4cd6-9065-528ef49d4a58 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:43.580193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dc4da-9218-4cd6-9065-528ef49d4a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:43.580466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dc4da-9218-4cd6-9065-528ef49d4a58 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:43.581155 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8dc4da-9218-4cd6-9065-528ef49d4a58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:43.582678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8dc4da-9218-4cd6-9065-528ef49d4a58 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:43.590125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 644/1301] 10.222.0.93 () {56 vars in 1087 bytes} [Fri Jun 3 15:16:43 2022] GET /identity/v3/services => generated 2191 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:16:45.944712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208778d8-0446-4f77-a6c3-55f7995fbf91 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:45.945239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208778d8-0446-4f77-a6c3-55f7995fbf91 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:45.945239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208778d8-0446-4f77-a6c3-55f7995fbf91 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:45.946507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 658/1302] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:16:45.952630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a966d56-76f9-4d7d-9e53-2b24217ba7a3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:45.953316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a966d56-76f9-4d7d-9e53-2b24217ba7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:45.953316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a966d56-76f9-4d7d-9e53-2b24217ba7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:45.971986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0a966d56-76f9-4d7d-9e53-2b24217ba7a3 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:16:45.973202 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0a966d56-76f9-4d7d-9e53-2b24217ba7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:16:46.057483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 645/1303] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:16:45 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 03 15:16:46.080722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208778d8-0446-4f77-a6c3-55f7995fbf91 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:46.190055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:46.197941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208778d8-0446-4f77-a6c3-55f7995fbf91 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:46.242592 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6403b-ddf5-4d45-bbb5-324b26236876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:46.243455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6403b-ddf5-4d45-bbb5-324b26236876 None admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:46.243787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6403b-ddf5-4d45-bbb5-324b26236876 None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:46.244134 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6403b-ddf5-4d45-bbb5-324b26236876 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:46.244832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6403b-ddf5-4d45-bbb5-324b26236876 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:46.246304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6403b-ddf5-4d45-bbb5-324b26236876 None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application [None req-a2f6403b-ddf5-4d45-bbb5-324b26236876 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 03 15:16:46.258717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 03 15:16:46.260094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.get_or_create( Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application with Lock( Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self._enter() Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application return self.creator() Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application created_value = creator( Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: ERROR keystone.server.flask.application Jun 03 15:16:46.261155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 659/1304] 10.222.0.93 () {56 vars in 1117 bytes} [Fri Jun 3 15:16:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 03 15:16:46.267905 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a966d56-76f9-4d7d-9e53-2b24217ba7a3 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:46.354270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:46.379201 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a966d56-76f9-4d7d-9e53-2b24217ba7a3 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:46.418212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c15a7-fc83-492b-8896-1b30da3d5480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:46.419339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c15a7-fc83-492b-8896-1b30da3d5480 None admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:46.419796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c15a7-fc83-492b-8896-1b30da3d5480 None admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:46.420110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c15a7-fc83-492b-8896-1b30da3d5480 None admin] PATH_INFO: `/v3/services` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:46.421194 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c15a7-fc83-492b-8896-1b30da3d5480 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:16:46.421820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c15a7-fc83-492b-8896-1b30da3d5480 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:46.424334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c15a7-fc83-492b-8896-1b30da3d5480 None admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:46.431743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 646/1305] 10.222.0.93 () {56 vars in 1116 bytes} [Fri Jun 3 15:16:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:16:46.439084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6403b-ddf5-4d45-bbb5-324b26236876 None admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:16:46.543996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6403b-ddf5-4d45-bbb5-324b26236876 None admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:16:46.579360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877f355-bda7-484a-9484-d35c02811b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:16:46.580018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877f355-bda7-484a-9484-d35c02811b6e None admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:16:46.580303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877f355-bda7-484a-9484-d35c02811b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:16:46.580580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877f355-bda7-484a-9484-d35c02811b6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:16:46.581319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877f355-bda7-484a-9484-d35c02811b6e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:16:46.582772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877f355-bda7-484a-9484-d35c02811b6e None admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:16:46.606935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 660/1306] 10.222.0.93 () {60 vars in 1144 bytes} [Fri Jun 3 15:16:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:16:56.428110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=30 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:16:56.602931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:17:54.533591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d330-0fae-4e94-a732-d8a23824242a None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:17:54.534664 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d330-0fae-4e94-a732-d8a23824242a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:17:54.535166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7d330-0fae-4e94-a732-d8a23824242a None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:17:54.540982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 647/1307] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:17:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:17:54.542943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb9c4e-d37f-426d-9354-828772f71fad None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:17:54.543494 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb9c4e-d37f-426d-9354-828772f71fad None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:17:54.543633 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb9c4e-d37f-426d-9354-828772f71fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:17:54.633567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-cfeb9c4e-d37f-426d-9354-828772f71fad None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:17:54.635018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cfeb9c4e-d37f-426d-9354-828772f71fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:17:54.844864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 661/1308] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:17:54.928791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66230727-3946-48cf-bb20-27ac34d06364 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:17:54.929783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66230727-3946-48cf-bb20-27ac34d06364 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:17:54.930599 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66230727-3946-48cf-bb20-27ac34d06364 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:17:54.934286 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 648/1309] 10.222.0.93 () {54 vars in 964 bytes} [Fri Jun 3 15:17:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:17:54.944294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d89407-b432-4d36-80ae-7925ab17b833 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:17:54.944917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d89407-b432-4d36-80ae-7925ab17b833 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:17:54.945424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d89407-b432-4d36-80ae-7925ab17b833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:17:55.249435 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-15d89407-b432-4d36-80ae-7925ab17b833 None None] MFA Rules not processed for user `e3f887d5bbcd424884537b95affd96f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:17:55.251079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-15d89407-b432-4d36-80ae-7925ab17b833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:17:55.280744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:17:55.411956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 662/1310] 10.222.0.93 () {58 vars in 1061 bytes} [Fri Jun 3 15:17:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:17:55.427155 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66230727-3946-48cf-bb20-27ac34d06364 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:17:55.430881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-66230727-3946-48cf-bb20-27ac34d06364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:17:55.677311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66230727-3946-48cf-bb20-27ac34d06364 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:17:55.814887 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80159435-506a-47af-b428-ce0ee3d32cd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:17:55.815993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80159435-506a-47af-b428-ce0ee3d32cd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:17:55.816492 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80159435-506a-47af-b428-ce0ee3d32cd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:17:55.816975 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80159435-506a-47af-b428-ce0ee3d32cd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:17:55.820157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-80159435-506a-47af-b428-ce0ee3d32cd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:17:55.919862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:17:56.012843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80159435-506a-47af-b428-ce0ee3d32cd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:17:56.015226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80159435-506a-47af-b428-ce0ee3d32cd2 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:17:56.038949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 649/1311] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:17:58.875439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48540bb-c37a-4018-8bf9-879908fc4f42 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:17:58.875963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48540bb-c37a-4018-8bf9-879908fc4f42 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:17:58.876078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48540bb-c37a-4018-8bf9-879908fc4f42 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:17:58.877504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 663/1312] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:17:58.882674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f80d7-f51b-46b8-9a12-fbee812d3089 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:17:58.883131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f80d7-f51b-46b8-9a12-fbee812d3089 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:17:58.883226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f80d7-f51b-46b8-9a12-fbee812d3089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:17:58.969949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-ab3f80d7-f51b-46b8-9a12-fbee812d3089 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:17:58.971121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ab3f80d7-f51b-46b8-9a12-fbee812d3089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:17:59.080220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 650/1313] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:17:59.119390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48540bb-c37a-4018-8bf9-879908fc4f42 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:17:59.121174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c48540bb-c37a-4018-8bf9-879908fc4f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:17:59.231240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48540bb-c37a-4018-8bf9-879908fc4f42 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:17:59.312613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:17:59.315461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc822406-aa81-4367-a9f4-f03e91692807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:17:59.316232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc822406-aa81-4367-a9f4-f03e91692807 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:17:59.316936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc822406-aa81-4367-a9f4-f03e91692807 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:17:59.317568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc822406-aa81-4367-a9f4-f03e91692807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:17:59.319649 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bc822406-aa81-4367-a9f4-f03e91692807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:17:59.438294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc822406-aa81-4367-a9f4-f03e91692807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:17:59.439786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc822406-aa81-4367-a9f4-f03e91692807 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:17:59.461374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 664/1314] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:02.502643 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5bc50-5c8e-466f-bf72-43ce11bea968 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:02.503188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5bc50-5c8e-466f-bf72-43ce11bea968 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:02.503688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5bc50-5c8e-466f-bf72-43ce11bea968 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:02.506423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 651/1315] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:02.512161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999899-be38-40c6-87f4-3e33affd49f2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:02.512670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999899-be38-40c6-87f4-3e33affd49f2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:02.512790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999899-be38-40c6-87f4-3e33affd49f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:02.584787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9f999899-be38-40c6-87f4-3e33affd49f2 None None] MFA Rules not processed for user `8d32be869c6c48259625a2926009e24a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:02.586005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9f999899-be38-40c6-87f4-3e33affd49f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:02.648903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:02.696063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 665/1316] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:02.753266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574c6f0-9dc6-4b81-8dc9-db211c9f371f None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:02.753266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574c6f0-9dc6-4b81-8dc9-db211c9f371f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:02.753266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0574c6f0-9dc6-4b81-8dc9-db211c9f371f None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:02.754220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 652/1317] 10.222.0.93 () {54 vars in 964 bytes} [Fri Jun 3 15:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:02.760684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b201d68-d445-40e7-bbe1-1c8f62e333ad None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:02.761014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b201d68-d445-40e7-bbe1-1c8f62e333ad None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:02.761328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b201d68-d445-40e7-bbe1-1c8f62e333ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:02.833900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3b201d68-d445-40e7-bbe1-1c8f62e333ad None None] MFA Rules not processed for user `e3f887d5bbcd424884537b95affd96f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:02.835140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3b201d68-d445-40e7-bbe1-1c8f62e333ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:02.927929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 666/1318] 10.222.0.93 () {58 vars in 1061 bytes} [Fri Jun 3 15:18:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:02.938204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574c6f0-9dc6-4b81-8dc9-db211c9f371f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:02.940069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0574c6f0-9dc6-4b81-8dc9-db211c9f371f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:02.955692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:03.115380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0574c6f0-9dc6-4b81-8dc9-db211c9f371f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:03.195334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56cf23-c727-438a-adf8-d8ba06f9de5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:03.195998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56cf23-c727-438a-adf8-d8ba06f9de5f service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:03.196277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56cf23-c727-438a-adf8-d8ba06f9de5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:03.196681 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56cf23-c727-438a-adf8-d8ba06f9de5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:03.198225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-af56cf23-c727-438a-adf8-d8ba06f9de5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:03.265520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:03.313422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56cf23-c727-438a-adf8-d8ba06f9de5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:03.314965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af56cf23-c727-438a-adf8-d8ba06f9de5f service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:03.334207 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 653/1319] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:18:06.609548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957b54-b8a0-45b4-beea-92988ccdddcf None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:06.610695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957b54-b8a0-45b4-beea-92988ccdddcf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:06.610695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba957b54-b8a0-45b4-beea-92988ccdddcf None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:06.611522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 667/1320] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:18:06.616939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929486-8608-42e8-abdc-19549923c6df None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:06.617390 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929486-8608-42e8-abdc-19549923c6df None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:06.617531 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95929486-8608-42e8-abdc-19549923c6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:06.682851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-95929486-8608-42e8-abdc-19549923c6df None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:06.684099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-95929486-8608-42e8-abdc-19549923c6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:06.781770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 654/1321] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:18:06.820019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba957b54-b8a0-45b4-beea-92988ccdddcf None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:06.821573 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ba957b54-b8a0-45b4-beea-92988ccdddcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:06.874032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:06.928447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba957b54-b8a0-45b4-beea-92988ccdddcf None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:07.002095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe103b7-c1ed-455e-847d-75e7d4cdf223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:07.002766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe103b7-c1ed-455e-847d-75e7d4cdf223 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:07.003051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe103b7-c1ed-455e-847d-75e7d4cdf223 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:07.003322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe103b7-c1ed-455e-847d-75e7d4cdf223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:07.004986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6fe103b7-c1ed-455e-847d-75e7d4cdf223 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:07.105873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:07.119857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe103b7-c1ed-455e-847d-75e7d4cdf223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:07.121358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe103b7-c1ed-455e-847d-75e7d4cdf223 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:07.141649 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 668/1322] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:10.180713 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eb4e5-b02d-4dd5-ab6a-6977384463de None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:10.181280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eb4e5-b02d-4dd5-ab6a-6977384463de None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:10.181280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eb4e5-b02d-4dd5-ab6a-6977384463de None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:10.182982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 655/1323] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:10.188374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b42c61-3854-465d-af65-c628ff844e79 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:10.188803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b42c61-3854-465d-af65-c628ff844e79 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:10.188919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b42c61-3854-465d-af65-c628ff844e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:10.256877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b8b42c61-3854-465d-af65-c628ff844e79 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:10.258075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b8b42c61-3854-465d-af65-c628ff844e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:10.357631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 669/1324] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:10.393640 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771eb4e5-b02d-4dd5-ab6a-6977384463de None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:10.395209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-771eb4e5-b02d-4dd5-ab6a-6977384463de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:10.415191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:10.505946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771eb4e5-b02d-4dd5-ab6a-6977384463de None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:10.586654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e51e9-e933-43c1-84db-56118984525c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:10.587538 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e51e9-e933-43c1-84db-56118984525c service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:10.587934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e51e9-e933-43c1-84db-56118984525c service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:10.588320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e51e9-e933-43c1-84db-56118984525c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:10.590176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-460e51e9-e933-43c1-84db-56118984525c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:10.671784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:10.718754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e51e9-e933-43c1-84db-56118984525c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:10.720203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e51e9-e933-43c1-84db-56118984525c service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:10.741290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 656/1325] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:13.064434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4a770-aeb1-46f3-83c8-d5df27cefa50 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:13.065008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4a770-aeb1-46f3-83c8-d5df27cefa50 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:13.065008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4a770-aeb1-46f3-83c8-d5df27cefa50 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:13.066699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 670/1326] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:13.076915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ff2e0-bbf7-4431-991a-a1b8fcfb96a9 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:13.077764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ff2e0-bbf7-4431-991a-a1b8fcfb96a9 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:13.077764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ff2e0-bbf7-4431-991a-a1b8fcfb96a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:13.186424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-405ff2e0-bbf7-4431-991a-a1b8fcfb96a9 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:13.188513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-405ff2e0-bbf7-4431-991a-a1b8fcfb96a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:13.315023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 657/1327] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:13.341027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4a770-aeb1-46f3-83c8-d5df27cefa50 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:13.344269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d6b4a770-aeb1-46f3-83c8-d5df27cefa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:13.435295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:13.487552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4a770-aeb1-46f3-83c8-d5df27cefa50 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:13.574232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c89b-ac10-46cb-892b-f36de0b53beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:13.574969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c89b-ac10-46cb-892b-f36de0b53beb service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:13.575538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c89b-ac10-46cb-892b-f36de0b53beb service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:13.575813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c89b-ac10-46cb-892b-f36de0b53beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:13.578143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a400c89b-ac10-46cb-892b-f36de0b53beb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:13.710178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:13.714158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400c89b-ac10-46cb-892b-f36de0b53beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:13.716711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400c89b-ac10-46cb-892b-f36de0b53beb service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:13.746221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 671/1328] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:16.203800 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb92a5-e99b-4f4c-af6c-459666b3d75b None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:16.204404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb92a5-e99b-4f4c-af6c-459666b3d75b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:16.204404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb92a5-e99b-4f4c-af6c-459666b3d75b None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:16.205747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 658/1329] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:16.211973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7739789-c844-4d17-890b-f16044e95d88 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:16.212486 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7739789-c844-4d17-890b-f16044e95d88 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:16.212606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7739789-c844-4d17-890b-f16044e95d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:16.290098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b7739789-c844-4d17-890b-f16044e95d88 None None] MFA Rules not processed for user `8d32be869c6c48259625a2926009e24a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:16.291335 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b7739789-c844-4d17-890b-f16044e95d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:16.413939 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 672/1330] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:16 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:18:16.452650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb92a5-e99b-4f4c-af6c-459666b3d75b None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:16.454292 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-91eb92a5-e99b-4f4c-af6c-459666b3d75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:16.507853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:16.591146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb92a5-e99b-4f4c-af6c-459666b3d75b None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:16.683878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebceddab-5e76-4d2e-ad63-04adde19328c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:16.684563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebceddab-5e76-4d2e-ad63-04adde19328c service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:16.684823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebceddab-5e76-4d2e-ad63-04adde19328c service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:16.685098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebceddab-5e76-4d2e-ad63-04adde19328c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:16.686824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ebceddab-5e76-4d2e-ad63-04adde19328c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:16.775576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:16.814467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebceddab-5e76-4d2e-ad63-04adde19328c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:16.816006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebceddab-5e76-4d2e-ad63-04adde19328c service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:16.835489 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 659/1331] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:19.537407 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1618f31-d17e-4104-962e-5af0a1cad0ce None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:19.537936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1618f31-d17e-4104-962e-5af0a1cad0ce None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:19.537936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1618f31-d17e-4104-962e-5af0a1cad0ce None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:19.539379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 673/1332] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:19.546446 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7c70f-0f80-49a9-b160-f1237ee712b7 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:19.546932 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7c70f-0f80-49a9-b160-f1237ee712b7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:19.547041 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a7c70f-0f80-49a9-b160-f1237ee712b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:19.628563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-08a7c70f-0f80-49a9-b160-f1237ee712b7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:19.629746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-08a7c70f-0f80-49a9-b160-f1237ee712b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:19.881250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 660/1333] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:19.921321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1618f31-d17e-4104-962e-5af0a1cad0ce None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:19.922933 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b1618f31-d17e-4104-962e-5af0a1cad0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:20.002753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:20.054568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1618f31-d17e-4104-962e-5af0a1cad0ce None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:20.162926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828a818-1813-42dd-9bee-4b9dc692576e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:20.163607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828a818-1813-42dd-9bee-4b9dc692576e service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:20.163859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828a818-1813-42dd-9bee-4b9dc692576e service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:20.164130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828a818-1813-42dd-9bee-4b9dc692576e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:20.165806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7828a818-1813-42dd-9bee-4b9dc692576e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:20.307738 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828a818-1813-42dd-9bee-4b9dc692576e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:20.309525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828a818-1813-42dd-9bee-4b9dc692576e service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:20.321197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:20.331709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 674/1334] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:22.707990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50f8b9-127e-4a6a-861b-54d1f03f9368 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:22.708554 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50f8b9-127e-4a6a-861b-54d1f03f9368 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:22.708554 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50f8b9-127e-4a6a-861b-54d1f03f9368 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:22.710005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 661/1335] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:22.715461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8325cd-7873-47cd-b913-f8449fb387a5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:22.715995 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8325cd-7873-47cd-b913-f8449fb387a5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:22.716110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8325cd-7873-47cd-b913-f8449fb387a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:22.785684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2e8325cd-7873-47cd-b913-f8449fb387a5 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:22.786969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2e8325cd-7873-47cd-b913-f8449fb387a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:22.901937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 675/1336] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:22.942721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50f8b9-127e-4a6a-861b-54d1f03f9368 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:22.944277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5c50f8b9-127e-4a6a-861b-54d1f03f9368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:22.991633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:23.068432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50f8b9-127e-4a6a-861b-54d1f03f9368 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:23.157647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b42bf-1625-4f3d-a8f1-2f38239c6773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:23.158305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b42bf-1625-4f3d-a8f1-2f38239c6773 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:23.158621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b42bf-1625-4f3d-a8f1-2f38239c6773 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:23.158888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b42bf-1625-4f3d-a8f1-2f38239c6773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:23.160563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-491b42bf-1625-4f3d-a8f1-2f38239c6773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:23.253918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:23.287080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b42bf-1625-4f3d-a8f1-2f38239c6773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:23.288528 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b42bf-1625-4f3d-a8f1-2f38239c6773 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:23.307584 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 662/1337] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:26.124304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc4e7e-28d5-4ec7-81fd-dab0b8cbd7df None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:26.125215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc4e7e-28d5-4ec7-81fd-dab0b8cbd7df None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:26.125215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc4e7e-28d5-4ec7-81fd-dab0b8cbd7df None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:26.126226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 676/1338] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:26.131730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38f49-655a-42b8-9f06-f98b6f74895a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:26.132206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38f49-655a-42b8-9f06-f98b6f74895a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:26.132300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc38f49-655a-42b8-9f06-f98b6f74895a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:26.230282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-5bc38f49-655a-42b8-9f06-f98b6f74895a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:26.231544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5bc38f49-655a-42b8-9f06-f98b6f74895a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:26.347484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 663/1339] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:26.383275 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc4e7e-28d5-4ec7-81fd-dab0b8cbd7df None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:26.385427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3dfc4e7e-28d5-4ec7-81fd-dab0b8cbd7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:26.488764 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:26.507183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc4e7e-28d5-4ec7-81fd-dab0b8cbd7df None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:26.512587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525dcb2b-be9a-4c99-80dc-2e105e0a2967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:26.513608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dcb2b-be9a-4c99-80dc-2e105e0a2967 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:26.513891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dcb2b-be9a-4c99-80dc-2e105e0a2967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:26.514167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dcb2b-be9a-4c99-80dc-2e105e0a2967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:26.526140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525dcb2b-be9a-4c99-80dc-2e105e0a2967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:26.527875 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525dcb2b-be9a-4c99-80dc-2e105e0a2967 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:26.538425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 677/1340] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:31.238334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d67f6-fc4f-45e3-9c11-62cf4172d1ed None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:31.239026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d67f6-fc4f-45e3-9c11-62cf4172d1ed None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:31.239026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d67f6-fc4f-45e3-9c11-62cf4172d1ed None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:31.240318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 664/1341] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:31.246444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c8f90-16c9-49d6-ba29-1bf6ba5f698c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:31.247072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c8f90-16c9-49d6-ba29-1bf6ba5f698c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:31.247309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c8f90-16c9-49d6-ba29-1bf6ba5f698c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:31.281853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c54c8f90-16c9-49d6-ba29-1bf6ba5f698c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:31.283278 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c54c8f90-16c9-49d6-ba29-1bf6ba5f698c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:31.373642 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 678/1342] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:31.414744 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d67f6-fc4f-45e3-9c11-62cf4172d1ed None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:31.416648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c91d67f6-fc4f-45e3-9c11-62cf4172d1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:31.430520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d67f6-fc4f-45e3-9c11-62cf4172d1ed None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:31.433988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e159a6-f868-4f41-8fc1-2ae554b02be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:31.434650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e159a6-f868-4f41-8fc1-2ae554b02be9 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:31.435095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e159a6-f868-4f41-8fc1-2ae554b02be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:31.435404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e159a6-f868-4f41-8fc1-2ae554b02be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:31.446278 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e159a6-f868-4f41-8fc1-2ae554b02be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:31.448026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e159a6-f868-4f41-8fc1-2ae554b02be9 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:31.459382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 665/1343] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:34.512434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b75f93-4f8f-4e12-a6df-81e40e9cb5aa None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:34.513049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b75f93-4f8f-4e12-a6df-81e40e9cb5aa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:34.513049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b75f93-4f8f-4e12-a6df-81e40e9cb5aa None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:34.514886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 679/1344] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:34.521020 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93249f-bd63-4a44-be2c-e87c6952ab56 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:34.521461 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93249f-bd63-4a44-be2c-e87c6952ab56 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:34.521808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93249f-bd63-4a44-be2c-e87c6952ab56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:34.545790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-7a93249f-bd63-4a44-be2c-e87c6952ab56 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:34.547032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7a93249f-bd63-4a44-be2c-e87c6952ab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:34.556997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 666/1345] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:34.596622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b75f93-4f8f-4e12-a6df-81e40e9cb5aa None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:34.607903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b75f93-4f8f-4e12-a6df-81e40e9cb5aa None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:34.611421 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d8563-4780-481e-b933-e3c3fa27a8b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:34.612017 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d8563-4780-481e-b933-e3c3fa27a8b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:34.612293 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d8563-4780-481e-b933-e3c3fa27a8b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:34.612581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d8563-4780-481e-b933-e3c3fa27a8b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:34.622081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d8563-4780-481e-b933-e3c3fa27a8b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:34.623547 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d8563-4780-481e-b933-e3c3fa27a8b8 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:34.633388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 680/1346] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:39.478253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cdd04-fee9-43b8-b787-84578f655e06 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:39.478913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cdd04-fee9-43b8-b787-84578f655e06 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:39.478913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cdd04-fee9-43b8-b787-84578f655e06 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:39.480704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 667/1347] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:39.487076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06c227-d1d8-4337-af6c-5d671ccd741c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:39.487606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06c227-d1d8-4337-af6c-5d671ccd741c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:39.487812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06c227-d1d8-4337-af6c-5d671ccd741c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:39.507342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6d06c227-d1d8-4337-af6c-5d671ccd741c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:39.508682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6d06c227-d1d8-4337-af6c-5d671ccd741c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:39.518966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 681/1348] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:39.559677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cdd04-fee9-43b8-b787-84578f655e06 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:39.577162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cdd04-fee9-43b8-b787-84578f655e06 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:39.581057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36833e-280d-4703-931b-2dbb0c59d467 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:39.581737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36833e-280d-4703-931b-2dbb0c59d467 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:39.582022 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36833e-280d-4703-931b-2dbb0c59d467 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:39.582300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36833e-280d-4703-931b-2dbb0c59d467 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:39.594294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36833e-280d-4703-931b-2dbb0c59d467 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:39.595974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36833e-280d-4703-931b-2dbb0c59d467 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:39.607072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 668/1349] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:45.520967 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a3f5-64aa-4850-8abf-c9f2b8b568fd None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:45.521620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a3f5-64aa-4850-8abf-c9f2b8b568fd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:45.522140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a3f5-64aa-4850-8abf-c9f2b8b568fd None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:45.523779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 682/1350] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:45.532245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78116b1c-ca17-40fe-bde6-07e220127395 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:45.532730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78116b1c-ca17-40fe-bde6-07e220127395 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:45.532844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78116b1c-ca17-40fe-bde6-07e220127395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:45.556571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:45.619818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-78116b1c-ca17-40fe-bde6-07e220127395 None None] MFA Rules not processed for user `50625c2b38ed41698ba6cccdb12730ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:45.621179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-78116b1c-ca17-40fe-bde6-07e220127395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:45.726225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 669/1351] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:18:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:49.501084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=57 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:50.298120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17899-935a-455d-9fa8-a261072b7c80 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:50.298597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17899-935a-455d-9fa8-a261072b7c80 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:50.298723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17899-935a-455d-9fa8-a261072b7c80 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:50.300316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 683/1352] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:50.305442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838cc91-daff-4d7f-96d8-bfad40b19bdc None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:50.305911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838cc91-daff-4d7f-96d8-bfad40b19bdc None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:50.306013 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838cc91-daff-4d7f-96d8-bfad40b19bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:50.326465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-7838cc91-daff-4d7f-96d8-bfad40b19bdc None None] MFA Rules not processed for user `50625c2b38ed41698ba6cccdb12730ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:50.328441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7838cc91-daff-4d7f-96d8-bfad40b19bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:50.337472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 670/1353] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:18:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:52.032986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c22cd-b33d-4225-8bb8-1cd38361ca1d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:52.033712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c22cd-b33d-4225-8bb8-1cd38361ca1d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:52.033712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c22cd-b33d-4225-8bb8-1cd38361ca1d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:52.034995 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 684/1354] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:52.040207 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706aaaa-3b06-473c-acba-5563cbd403d3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:52.040677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706aaaa-3b06-473c-acba-5563cbd403d3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:52.040774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706aaaa-3b06-473c-acba-5563cbd403d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:52.060216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-1706aaaa-3b06-473c-acba-5563cbd403d3 None None] MFA Rules not processed for user `50625c2b38ed41698ba6cccdb12730ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:52.062021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1706aaaa-3b06-473c-acba-5563cbd403d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:52.071551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 671/1355] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:18:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:56.142314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c12aa-9845-4eae-b697-c544936a1128 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:56.142973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c12aa-9845-4eae-b697-c544936a1128 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:56.143267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c12aa-9845-4eae-b697-c544936a1128 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:56.144869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 685/1356] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:56.150998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea0a4-09f6-49c0-a856-d2894cc84673 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:56.151880 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea0a4-09f6-49c0-a856-d2894cc84673 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:56.152153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea0a4-09f6-49c0-a856-d2894cc84673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:56.185323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-07dea0a4-09f6-49c0-a856-d2894cc84673 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:56.186566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-07dea0a4-09f6-49c0-a856-d2894cc84673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:56.197729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 672/1357] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:56.259311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8030f1-aee4-4bb1-9b64-d4d1e214f438 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:56.259602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8030f1-aee4-4bb1-9b64-d4d1e214f438 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:56.259883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8030f1-aee4-4bb1-9b64-d4d1e214f438 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:56.261239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 686/1358] 10.222.0.93 () {54 vars in 961 bytes} [Fri Jun 3 15:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:56.269481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb41b1b-d365-4ff9-a049-104b240d9bd6 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:56.269899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb41b1b-d365-4ff9-a049-104b240d9bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:56.270335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb41b1b-d365-4ff9-a049-104b240d9bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:56.334503 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-beb41b1b-d365-4ff9-a049-104b240d9bd6 None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:56.335923 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-beb41b1b-d365-4ff9-a049-104b240d9bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:56.402451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:18:56.435498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 673/1359] 10.222.0.93 () {58 vars in 1058 bytes} [Fri Jun 3 15:18:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:56.443685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8030f1-aee4-4bb1-9b64-d4d1e214f438 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:56.462926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8030f1-aee4-4bb1-9b64-d4d1e214f438 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:56.470202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8556069-c135-49d6-9afa-5b14afc12bdb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:56.471014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8556069-c135-49d6-9afa-5b14afc12bdb service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:56.471729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8556069-c135-49d6-9afa-5b14afc12bdb service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:56.472259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8556069-c135-49d6-9afa-5b14afc12bdb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:56.484811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8556069-c135-49d6-9afa-5b14afc12bdb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:56.486603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8556069-c135-49d6-9afa-5b14afc12bdb service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:56.504132 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 687/1360] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:18:59.088903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823093e-4d45-4d6c-ac80-a584095053f1 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:59.089492 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823093e-4d45-4d6c-ac80-a584095053f1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:59.089492 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823093e-4d45-4d6c-ac80-a584095053f1 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:59.091828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 674/1361] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:18:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:18:59.096510 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590435-0ecf-4344-8fb1-06542a48a882 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:59.097026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590435-0ecf-4344-8fb1-06542a48a882 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:59.097178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41590435-0ecf-4344-8fb1-06542a48a882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:59.117074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-41590435-0ecf-4344-8fb1-06542a48a882 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:18:59.118341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-41590435-0ecf-4344-8fb1-06542a48a882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:18:59.128286 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 688/1362] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:18:59.175011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823093e-4d45-4d6c-ac80-a584095053f1 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:18:59.193870 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823093e-4d45-4d6c-ac80-a584095053f1 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:18:59.203249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9b086d-f03f-4258-8300-cc7426d08fa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:18:59.206653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b086d-f03f-4258-8300-cc7426d08fa4 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:18:59.206653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b086d-f03f-4258-8300-cc7426d08fa4 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:18:59.206653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9b086d-f03f-4258-8300-cc7426d08fa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:18:59.218365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b086d-f03f-4258-8300-cc7426d08fa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:18:59.221873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9b086d-f03f-4258-8300-cc7426d08fa4 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:18:59.236883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 675/1363] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:01.921863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad1735-a6e2-4438-8236-36c42032c6b1 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:01.922367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad1735-a6e2-4438-8236-36c42032c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:01.922367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad1735-a6e2-4438-8236-36c42032c6b1 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:01.923702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 689/1364] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:01.929097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef1593-bd29-4872-9e85-dba0dc40d752 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:01.929519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef1593-bd29-4872-9e85-dba0dc40d752 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:01.929614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef1593-bd29-4872-9e85-dba0dc40d752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:01.956379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-97ef1593-bd29-4872-9e85-dba0dc40d752 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:01.957685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-97ef1593-bd29-4872-9e85-dba0dc40d752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:01.966212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 676/1365] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:02.015561 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba65e3-1afe-4226-8fc8-d16584dcf350 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:02.016080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba65e3-1afe-4226-8fc8-d16584dcf350 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:02.016364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba65e3-1afe-4226-8fc8-d16584dcf350 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:02.017793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 690/1366] 10.222.0.93 () {54 vars in 961 bytes} [Fri Jun 3 15:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:02.025122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a894600-9389-4698-806a-b3f708641163 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:02.025406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a894600-9389-4698-806a-b3f708641163 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:02.025713 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a894600-9389-4698-806a-b3f708641163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:02.046289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-8a894600-9389-4698-806a-b3f708641163 None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:02.047645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8a894600-9389-4698-806a-b3f708641163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:02.057231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 677/1367] 10.222.0.93 () {58 vars in 1058 bytes} [Fri Jun 3 15:19:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:02.066086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba65e3-1afe-4226-8fc8-d16584dcf350 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:02.081011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba65e3-1afe-4226-8fc8-d16584dcf350 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:02.087943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908aa96-2405-4af0-a35e-1d816482de32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:02.088615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908aa96-2405-4af0-a35e-1d816482de32 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:02.089215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908aa96-2405-4af0-a35e-1d816482de32 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:02.089700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908aa96-2405-4af0-a35e-1d816482de32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:02.099101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908aa96-2405-4af0-a35e-1d816482de32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:02.100705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908aa96-2405-4af0-a35e-1d816482de32 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:02.115521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 691/1368] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:04.679175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453e340-8f3e-4070-b992-b01d90309a57 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:04.679839 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453e340-8f3e-4070-b992-b01d90309a57 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:04.679839 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453e340-8f3e-4070-b992-b01d90309a57 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:04.681556 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 678/1369] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:04.689591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de74dcc-0524-4388-8282-40642d7e4424 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:04.690580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de74dcc-0524-4388-8282-40642d7e4424 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:04.690580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de74dcc-0524-4388-8282-40642d7e4424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:04.714888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7de74dcc-0524-4388-8282-40642d7e4424 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:04.716473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7de74dcc-0524-4388-8282-40642d7e4424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:04.726098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 692/1370] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:19:04.771816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453e340-8f3e-4070-b992-b01d90309a57 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:04.794475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453e340-8f3e-4070-b992-b01d90309a57 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:04.803586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2393a3b-693a-483f-a845-94518ff64855 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:04.804668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2393a3b-693a-483f-a845-94518ff64855 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:04.805339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2393a3b-693a-483f-a845-94518ff64855 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:04.805820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2393a3b-693a-483f-a845-94518ff64855 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:04.822922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2393a3b-693a-483f-a845-94518ff64855 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:04.825856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2393a3b-693a-483f-a845-94518ff64855 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:04.849332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 679/1371] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:05.151060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489c9e8-4f86-49d9-b38d-46b82956571f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:05.151324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489c9e8-4f86-49d9-b38d-46b82956571f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:05.151833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489c9e8-4f86-49d9-b38d-46b82956571f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:05.153250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 693/1372] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:19:05.159431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be34d-8c04-4f80-9642-8ac27f312679 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:05.160055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be34d-8c04-4f80-9642-8ac27f312679 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:05.160562 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0be34d-8c04-4f80-9642-8ac27f312679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:05.182850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-1d0be34d-8c04-4f80-9642-8ac27f312679 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:05.184324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1d0be34d-8c04-4f80-9642-8ac27f312679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:05.196613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 680/1373] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:05.533238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3488d0-f5e3-4204-b1e6-5da591702ad6 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:05.533238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3488d0-f5e3-4204-b1e6-5da591702ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:05.533662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3488d0-f5e3-4204-b1e6-5da591702ad6 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:05.535353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 694/1374] 10.222.0.93 () {54 vars in 963 bytes} [Fri Jun 3 15:19:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:05.540796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc0c8-ba56-423e-8992-4c4cf614bd68 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:05.541271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc0c8-ba56-423e-8992-4c4cf614bd68 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:05.541556 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fc0c8-ba56-423e-8992-4c4cf614bd68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:05.607417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0d3fc0c8-ba56-423e-8992-4c4cf614bd68 None None] MFA Rules not processed for user `98aa51c4bc604b0ca8a6c69bc1455bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:05.609352 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0d3fc0c8-ba56-423e-8992-4c4cf614bd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:05.713844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 681/1375] 10.222.0.93 () {58 vars in 1060 bytes} [Fri Jun 3 15:19:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:05.725527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3488d0-f5e3-4204-b1e6-5da591702ad6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:05.741458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3488d0-f5e3-4204-b1e6-5da591702ad6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:05.746335 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f969d8-025c-4484-aad4-793359370654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:05.747023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f969d8-025c-4484-aad4-793359370654 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:05.747305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f969d8-025c-4484-aad4-793359370654 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:05.747584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f969d8-025c-4484-aad4-793359370654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:05.762132 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f969d8-025c-4484-aad4-793359370654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:05.765833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f969d8-025c-4484-aad4-793359370654 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:05.778620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 695/1376] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:07.565307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd891869-9cc5-454f-aaeb-7a2d9ff8bdda None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:07.565848 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd891869-9cc5-454f-aaeb-7a2d9ff8bdda None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:07.565994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd891869-9cc5-454f-aaeb-7a2d9ff8bdda None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:07.567435 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 682/1377] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:07.572918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af223611-d917-4f6a-82e9-30b9f69d31be None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:07.573397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af223611-d917-4f6a-82e9-30b9f69d31be None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:07.573511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af223611-d917-4f6a-82e9-30b9f69d31be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:07.592567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-af223611-d917-4f6a-82e9-30b9f69d31be None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:07.593899 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-af223611-d917-4f6a-82e9-30b9f69d31be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:07.601931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 696/1378] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:07.647860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd891869-9cc5-454f-aaeb-7a2d9ff8bdda None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:07.666214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd891869-9cc5-454f-aaeb-7a2d9ff8bdda None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:07.672251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e3b14-029a-401c-a568-d820d51f8b70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:07.672990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e3b14-029a-401c-a568-d820d51f8b70 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:07.673345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e3b14-029a-401c-a568-d820d51f8b70 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:07.673691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e3b14-029a-401c-a568-d820d51f8b70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:07.682823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e3b14-029a-401c-a568-d820d51f8b70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:07.684434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7e3b14-029a-401c-a568-d820d51f8b70 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:07.697177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 683/1379] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:10.161160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4429-2be8-40bc-b4f3-d2efd3cfabfe None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:10.161975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4429-2be8-40bc-b4f3-d2efd3cfabfe None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:10.161975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf4429-2be8-40bc-b4f3-d2efd3cfabfe None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:10.163184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 697/1380] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:19:10.170106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00917b33-83ed-463b-b4f6-489ef67643cf None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:10.170106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00917b33-83ed-463b-b4f6-489ef67643cf None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:10.170106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00917b33-83ed-463b-b4f6-489ef67643cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:10.192428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-00917b33-83ed-463b-b4f6-489ef67643cf None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:10.193670 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-00917b33-83ed-463b-b4f6-489ef67643cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:10.202738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 684/1381] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:10.250136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf4429-2be8-40bc-b4f3-d2efd3cfabfe None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:10.266390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf4429-2be8-40bc-b4f3-d2efd3cfabfe None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:10.274350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77865323-cd94-49f2-b7f9-68aa5927e933 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:10.275494 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865323-cd94-49f2-b7f9-68aa5927e933 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:10.276218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865323-cd94-49f2-b7f9-68aa5927e933 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:10.276988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77865323-cd94-49f2-b7f9-68aa5927e933 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:10.288261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77865323-cd94-49f2-b7f9-68aa5927e933 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:10.290254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77865323-cd94-49f2-b7f9-68aa5927e933 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:10.303373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 698/1382] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:19:12.619258 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b62f8-fc29-4085-83ff-b6b1d16c0741 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:12.619735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b62f8-fc29-4085-83ff-b6b1d16c0741 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:12.619835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b62f8-fc29-4085-83ff-b6b1d16c0741 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:12.621163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 685/1383] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:12.627182 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53143d-04e5-4314-9ed3-4c9f4d6b5b27 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:12.627554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53143d-04e5-4314-9ed3-4c9f4d6b5b27 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:12.627704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53143d-04e5-4314-9ed3-4c9f4d6b5b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:12.647357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3d53143d-04e5-4314-9ed3-4c9f4d6b5b27 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:12.648641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3d53143d-04e5-4314-9ed3-4c9f4d6b5b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:12.657880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 699/1384] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:12.700275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b62f8-fc29-4085-83ff-b6b1d16c0741 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:12.715866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b62f8-fc29-4085-83ff-b6b1d16c0741 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:12.722971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c902d0-0f1a-46a9-a58c-a30bff05a045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:12.723617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c902d0-0f1a-46a9-a58c-a30bff05a045 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:12.723896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c902d0-0f1a-46a9-a58c-a30bff05a045 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:12.724170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c902d0-0f1a-46a9-a58c-a30bff05a045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:12.733766 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c902d0-0f1a-46a9-a58c-a30bff05a045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:12.735806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c902d0-0f1a-46a9-a58c-a30bff05a045 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:12.746001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 686/1385] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:15.043691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225234a-b97c-42be-8c65-0cf0124499cf None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:15.050455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225234a-b97c-42be-8c65-0cf0124499cf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:15.050455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225234a-b97c-42be-8c65-0cf0124499cf None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:15.050455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 700/1386] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:19:15.051488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce02bb2-2052-4cc3-ad19-9b25cc090f7e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:15.051904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce02bb2-2052-4cc3-ad19-9b25cc090f7e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:15.052043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce02bb2-2052-4cc3-ad19-9b25cc090f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:15.074072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-fce02bb2-2052-4cc3-ad19-9b25cc090f7e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:15.075447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-fce02bb2-2052-4cc3-ad19-9b25cc090f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:15.085617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 687/1387] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:15.138450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225234a-b97c-42be-8c65-0cf0124499cf None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:15.154276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225234a-b97c-42be-8c65-0cf0124499cf None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:15.160268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b2f86-c9a9-4899-aa45-8d7d269e5579 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:15.160919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b2f86-c9a9-4899-aa45-8d7d269e5579 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:15.161204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b2f86-c9a9-4899-aa45-8d7d269e5579 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:15.161523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b2f86-c9a9-4899-aa45-8d7d269e5579 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:15.171308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b2f86-c9a9-4899-aa45-8d7d269e5579 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:15.173420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b2f86-c9a9-4899-aa45-8d7d269e5579 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:15.189259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 701/1388] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:19:17.614391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2e82f-9faf-402d-a3c6-86f758728f64 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:17.614983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2e82f-9faf-402d-a3c6-86f758728f64 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:17.615147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2e82f-9faf-402d-a3c6-86f758728f64 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:17.617027 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 688/1389] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:17.622756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b2055-8e0e-4624-99af-25f2c6d4d0ff None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:17.623186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b2055-8e0e-4624-99af-25f2c6d4d0ff None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:17.623317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b2055-8e0e-4624-99af-25f2c6d4d0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:17.641437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a72b2055-8e0e-4624-99af-25f2c6d4d0ff None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:17.642668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a72b2055-8e0e-4624-99af-25f2c6d4d0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:17.653972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 702/1390] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:17.704268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2e82f-9faf-402d-a3c6-86f758728f64 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:17.719422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2e82f-9faf-402d-a3c6-86f758728f64 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:17.724992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ecf2d6-6d32-4b0c-81fd-edabed774cee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:17.725597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecf2d6-6d32-4b0c-81fd-edabed774cee service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:17.725884 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecf2d6-6d32-4b0c-81fd-edabed774cee service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:17.726155 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ecf2d6-6d32-4b0c-81fd-edabed774cee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:17.734426 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecf2d6-6d32-4b0c-81fd-edabed774cee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:17.736048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ecf2d6-6d32-4b0c-81fd-edabed774cee service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:17.745430 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 689/1391] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:20.161808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55bc79-7bbd-4984-8361-5018d75df0dd None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:20.162350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55bc79-7bbd-4984-8361-5018d75df0dd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:20.162475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55bc79-7bbd-4984-8361-5018d75df0dd None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:20.163826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 703/1392] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:20.169276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73fde9-5be5-462c-8081-778ede2f7285 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:20.169815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73fde9-5be5-462c-8081-778ede2f7285 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:20.169815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73fde9-5be5-462c-8081-778ede2f7285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:20.195412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-1b73fde9-5be5-462c-8081-778ede2f7285 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:20.196651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1b73fde9-5be5-462c-8081-778ede2f7285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:20.204945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 690/1393] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:20.251133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55bc79-7bbd-4984-8361-5018d75df0dd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:20.259449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=46 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:19:20.267806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55bc79-7bbd-4984-8361-5018d75df0dd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:20.273497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a0ee4-4910-4b9d-9c2f-da0b07d064e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:20.274140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0ee4-4910-4b9d-9c2f-da0b07d064e3 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:20.274427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0ee4-4910-4b9d-9c2f-da0b07d064e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:20.274751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a0ee4-4910-4b9d-9c2f-da0b07d064e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:20.283768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a0ee4-4910-4b9d-9c2f-da0b07d064e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:20.285368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a0ee4-4910-4b9d-9c2f-da0b07d064e3 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:20.295266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 704/1394] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:22.626060 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162b956-669b-47d5-a39f-65ff55de1802 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:22.626760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162b956-669b-47d5-a39f-65ff55de1802 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:22.626760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7162b956-669b-47d5-a39f-65ff55de1802 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:22.628083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 691/1395] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:22.633303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae644d-42ce-4b85-b453-894d4192fc8f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:22.633731 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae644d-42ce-4b85-b453-894d4192fc8f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:22.633940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae644d-42ce-4b85-b453-894d4192fc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:22.652582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1eae644d-42ce-4b85-b453-894d4192fc8f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:22.654062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1eae644d-42ce-4b85-b453-894d4192fc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:22.663027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 705/1396] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:22.707509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162b956-669b-47d5-a39f-65ff55de1802 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:22.722910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7162b956-669b-47d5-a39f-65ff55de1802 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:22.728460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ba64c-6b15-430f-b70e-a01af155ca7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:22.729098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ba64c-6b15-430f-b70e-a01af155ca7f service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:22.729382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ba64c-6b15-430f-b70e-a01af155ca7f service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:22.729657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ba64c-6b15-430f-b70e-a01af155ca7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:22.739957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ba64c-6b15-430f-b70e-a01af155ca7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:22.741534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ba64c-6b15-430f-b70e-a01af155ca7f service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:22.751337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 692/1397] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:22.814103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7458f-aee9-40f3-a50d-e070b1e19336 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:22.814395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7458f-aee9-40f3-a50d-e070b1e19336 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:22.814701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7458f-aee9-40f3-a50d-e070b1e19336 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:22.816240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 706/1398] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:22.822703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f81e0-5572-4173-85bf-b718b1be4396 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:22.822703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f81e0-5572-4173-85bf-b718b1be4396 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:22.822703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f81e0-5572-4173-85bf-b718b1be4396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:22.835967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:19:22.843042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d32f81e0-5572-4173-85bf-b718b1be4396 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:22.844299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d32f81e0-5572-4173-85bf-b718b1be4396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:22.852866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 693/1399] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:19:22.915296 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a3416-95e0-4dd7-a540-850544ec3c33 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:22.915582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a3416-95e0-4dd7-a540-850544ec3c33 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:22.915861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725a3416-95e0-4dd7-a540-850544ec3c33 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:22.917279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 707/1400] 10.222.0.93 () {54 vars in 962 bytes} [Fri Jun 3 15:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:22.922724 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f2dfc-beb0-4bcc-b404-452d8cca4889 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:22.923013 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f2dfc-beb0-4bcc-b404-452d8cca4889 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:22.923296 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f2dfc-beb0-4bcc-b404-452d8cca4889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:22.978066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-745f2dfc-beb0-4bcc-b404-452d8cca4889 None None] MFA Rules not processed for user `13121a263c31433dbd4a5082b4bf3bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:22.979328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-745f2dfc-beb0-4bcc-b404-452d8cca4889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:23.043549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 694/1401] 10.222.0.93 () {58 vars in 1059 bytes} [Fri Jun 3 15:19:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:23.051620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a3416-95e0-4dd7-a540-850544ec3c33 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:23.062378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725a3416-95e0-4dd7-a540-850544ec3c33 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:23.065986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8532d0-8a9e-4010-b104-5887e4eab181 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13121a263c31433dbd4a5082b4bf3bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:23.066659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8532d0-8a9e-4010-b104-5887e4eab181 service glance] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:23.067035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8532d0-8a9e-4010-b104-5887e4eab181 service glance] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:23.067519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8532d0-8a9e-4010-b104-5887e4eab181 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:23.078765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8532d0-8a9e-4010-b104-5887e4eab181 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:23.080217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8532d0-8a9e-4010-b104-5887e4eab181 service glance] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:23.090650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 708/1402] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:23.633512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59124d4a-957b-430b-acf4-526808f7e461 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:23.634020 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59124d4a-957b-430b-acf4-526808f7e461 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:23.634126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59124d4a-957b-430b-acf4-526808f7e461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:23.692107 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-59124d4a-957b-430b-acf4-526808f7e461 None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:23.693460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-59124d4a-957b-430b-acf4-526808f7e461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:23.774899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 695/1403] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:19:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:23.834816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd2211-1427-40b1-8e57-e0e69afb52c9 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:23.835648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd2211-1427-40b1-8e57-e0e69afb52c9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:23.835648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd2211-1427-40b1-8e57-e0e69afb52c9 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:23.837101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 709/1404] 10.222.0.93 () {54 vars in 942 bytes} [Fri Jun 3 15:19:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:23.843936 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263998e0-1d47-4164-9bd6-75ec660306f3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:23.844441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263998e0-1d47-4164-9bd6-75ec660306f3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:23.844994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263998e0-1d47-4164-9bd6-75ec660306f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:23.907631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-263998e0-1d47-4164-9bd6-75ec660306f3 None None] MFA Rules not processed for user `bb4fb8f859dc4489b8c4fd8fdc09c3f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:23.908969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-263998e0-1d47-4164-9bd6-75ec660306f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:23.930958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:19:24.003314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 696/1405] 10.222.0.93 () {58 vars in 1039 bytes} [Fri Jun 3 15:19:23 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:24.012858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd2211-1427-40b1-8e57-e0e69afb52c9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:24.034200 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd2211-1427-40b1-8e57-e0e69afb52c9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:24.039300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22273e-b35f-461e-b4dc-c8fe7c639680 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:24.040054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22273e-b35f-461e-b4dc-c8fe7c639680 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:24.040414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22273e-b35f-461e-b4dc-c8fe7c639680 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:24.042081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22273e-b35f-461e-b4dc-c8fe7c639680 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:24.050199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22273e-b35f-461e-b4dc-c8fe7c639680 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:24.051840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22273e-b35f-461e-b4dc-c8fe7c639680 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:24.061592 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 710/1406] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:19:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:19:25.181073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44ec9a-5bc0-4f3a-aef8-42bda4805d2e None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:25.181665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44ec9a-5bc0-4f3a-aef8-42bda4805d2e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:25.182032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44ec9a-5bc0-4f3a-aef8-42bda4805d2e None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:25.188216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 697/1407] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:19:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:25.194090 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e45d3-85c3-4b86-9465-4fe5941c1449 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:25.195521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e45d3-85c3-4b86-9465-4fe5941c1449 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:25.196039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e45d3-85c3-4b86-9465-4fe5941c1449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:25.215435 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-109e45d3-85c3-4b86-9465-4fe5941c1449 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:25.216730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-109e45d3-85c3-4b86-9465-4fe5941c1449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:25.234305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 711/1408] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:25.280162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44ec9a-5bc0-4f3a-aef8-42bda4805d2e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:25.299579 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab44ec9a-5bc0-4f3a-aef8-42bda4805d2e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:25.307121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0144683-17d3-4006-98ef-fe93fbacb92d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:25.309479 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0144683-17d3-4006-98ef-fe93fbacb92d service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:25.310046 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0144683-17d3-4006-98ef-fe93fbacb92d service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:25.310745 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0144683-17d3-4006-98ef-fe93fbacb92d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:25.330903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0144683-17d3-4006-98ef-fe93fbacb92d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:25.333291 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0144683-17d3-4006-98ef-fe93fbacb92d service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:25.345152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 698/1409] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:19:35.210008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=23 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:19:35.338934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=28 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:19:48.508523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025fe84-e6d1-410a-ac76-8a3f2899217f None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:48.509194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025fe84-e6d1-410a-ac76-8a3f2899217f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:48.509194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025fe84-e6d1-410a-ac76-8a3f2899217f None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:48.510647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 712/1410] 10.222.0.93 () {54 vars in 918 bytes} [Fri Jun 3 15:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:48.516479 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697a61-c1dc-4852-b559-030bdaf484b3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:48.517021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697a61-c1dc-4852-b559-030bdaf484b3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:48.517021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd697a61-c1dc-4852-b559-030bdaf484b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:48.536428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-dd697a61-c1dc-4852-b559-030bdaf484b3 None None] MFA Rules not processed for user `50625c2b38ed41698ba6cccdb12730ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:48.537955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-dd697a61-c1dc-4852-b559-030bdaf484b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:48.547071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 699/1411] 10.222.0.93 () {58 vars in 1015 bytes} [Fri Jun 3 15:19:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:48.561576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d8888-c7b6-4f93-ad3d-38646c253509 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:48.561853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d8888-c7b6-4f93-ad3d-38646c253509 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:48.562124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d8888-c7b6-4f93-ad3d-38646c253509 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:48.563571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 713/1412] 10.222.0.93 () {54 vars in 918 bytes} [Fri Jun 3 15:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:48.568627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70a529-f20c-44e0-8acc-e87e411775f1 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:48.569007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70a529-f20c-44e0-8acc-e87e411775f1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:48.569309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70a529-f20c-44e0-8acc-e87e411775f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:48.588619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-1d70a529-f20c-44e0-8acc-e87e411775f1 None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:48.589814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1d70a529-f20c-44e0-8acc-e87e411775f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:48.599503 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 700/1413] 10.222.0.93 () {58 vars in 1015 bytes} [Fri Jun 3 15:19:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:51.337929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e919a-42f4-4417-ad67-59eb26c1bc11 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:51.338610 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e919a-42f4-4417-ad67-59eb26c1bc11 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:51.338610 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e919a-42f4-4417-ad67-59eb26c1bc11 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:51.339878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:51.345701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de9c17-d14b-4925-989c-42ef3ce00e5b None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:51.346111 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de9c17-d14b-4925-989c-42ef3ce00e5b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:51.346245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de9c17-d14b-4925-989c-42ef3ce00e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:51.363635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-a3de9c17-d14b-4925-989c-42ef3ce00e5b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:51.364820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a3de9c17-d14b-4925-989c-42ef3ce00e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:51.372416 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:51.415741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e919a-42f4-4417-ad67-59eb26c1bc11 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:51.430598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e919a-42f4-4417-ad67-59eb26c1bc11 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:51.435899 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9c344-9eca-45be-89e7-1c44b382ba8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:51.436529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9c344-9eca-45be-89e7-1c44b382ba8b service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:51.436810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9c344-9eca-45be-89e7-1c44b382ba8b service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:51.437082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9c344-9eca-45be-89e7-1c44b382ba8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:51.445077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9c344-9eca-45be-89e7-1c44b382ba8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:51.446691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9c344-9eca-45be-89e7-1c44b382ba8b service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:51.456425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 715/1416] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:52.462687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896e932-7d33-4dda-a665-9997d8cdc297 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:52.463582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896e932-7d33-4dda-a665-9997d8cdc297 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:52.463778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896e932-7d33-4dda-a665-9997d8cdc297 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:52.465267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:19:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:52.471042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c936b8-230d-4bb6-8cce-82ce47c862ff None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:52.471524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c936b8-230d-4bb6-8cce-82ce47c862ff None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:52.471629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c936b8-230d-4bb6-8cce-82ce47c862ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:52.494409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-75c936b8-230d-4bb6-8cce-82ce47c862ff None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:52.495669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-75c936b8-230d-4bb6-8cce-82ce47c862ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:52.503683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:52.680951 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896e932-7d33-4dda-a665-9997d8cdc297 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:52.696705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896e932-7d33-4dda-a665-9997d8cdc297 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:52.702628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3fd70-b85d-4910-a93e-51a8278c5b25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:52.703285 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3fd70-b85d-4910-a93e-51a8278c5b25 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:52.703563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3fd70-b85d-4910-a93e-51a8278c5b25 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:52.703839 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3fd70-b85d-4910-a93e-51a8278c5b25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:52.712648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3fd70-b85d-4910-a93e-51a8278c5b25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:52.714314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3fd70-b85d-4910-a93e-51a8278c5b25 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:52.724743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 703/1419] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:53.685278 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e3b77-9f0e-4aa8-8354-27a59543505d None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:53.685934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e3b77-9f0e-4aa8-8354-27a59543505d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:53.685934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e3b77-9f0e-4aa8-8354-27a59543505d None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:53.687358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:53.692602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d93c-ed36-40d8-90bb-d78b12d2f4d3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:53.693031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d93c-ed36-40d8-90bb-d78b12d2f4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:53.693158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7d93c-ed36-40d8-90bb-d78b12d2f4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:53.712209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-a2e7d93c-ed36-40d8-90bb-d78b12d2f4d3 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:53.713613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a2e7d93c-ed36-40d8-90bb-d78b12d2f4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:53.722548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:53.872349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e3b77-9f0e-4aa8-8354-27a59543505d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:53.887964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e3b77-9f0e-4aa8-8354-27a59543505d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:53.893950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d99cc1-6d0e-4b0d-aa41-b155aaf3080c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:53.894645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d99cc1-6d0e-4b0d-aa41-b155aaf3080c service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:53.894927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d99cc1-6d0e-4b0d-aa41-b155aaf3080c service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:53.895191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d99cc1-6d0e-4b0d-aa41-b155aaf3080c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:53.904664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d99cc1-6d0e-4b0d-aa41-b155aaf3080c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:53.906549 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d99cc1-6d0e-4b0d-aa41-b155aaf3080c service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:53.919863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 718/1422] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:54.859518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67730db2-250a-41f1-8118-c6ffe2b4c866 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:54.860088 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67730db2-250a-41f1-8118-c6ffe2b4c866 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:54.860088 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67730db2-250a-41f1-8118-c6ffe2b4c866 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:54.861468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:19:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:54.866314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab57c6-1616-481a-90b9-20c8bf3adbdc None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:54.866745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab57c6-1616-481a-90b9-20c8bf3adbdc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:54.866923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab57c6-1616-481a-90b9-20c8bf3adbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:54.884461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-89ab57c6-1616-481a-90b9-20c8bf3adbdc None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:54.885872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-89ab57c6-1616-481a-90b9-20c8bf3adbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:54.893956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:55.043742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67730db2-250a-41f1-8118-c6ffe2b4c866 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:55.057873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67730db2-250a-41f1-8118-c6ffe2b4c866 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:55.063287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4788f33-5826-427a-919c-9f3ea1d9513e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:55.063917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4788f33-5826-427a-919c-9f3ea1d9513e service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:55.064199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4788f33-5826-427a-919c-9f3ea1d9513e service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:55.064477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4788f33-5826-427a-919c-9f3ea1d9513e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:55.072388 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4788f33-5826-427a-919c-9f3ea1d9513e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:55.073993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4788f33-5826-427a-919c-9f3ea1d9513e service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:55.084175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 706/1425] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:55.994308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c290f9e-edf5-4c97-832c-cd3f3572bbe4 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:55.994889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c290f9e-edf5-4c97-832c-cd3f3572bbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:55.994889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c290f9e-edf5-4c97-832c-cd3f3572bbe4 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:55.996237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:56.001894 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37087c6c-7ec2-4816-8897-2c56888d9de9 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:56.002352 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37087c6c-7ec2-4816-8897-2c56888d9de9 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:56.002448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37087c6c-7ec2-4816-8897-2c56888d9de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:56.020163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-37087c6c-7ec2-4816-8897-2c56888d9de9 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:56.021361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-37087c6c-7ec2-4816-8897-2c56888d9de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:56.032457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:19:56.182449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c290f9e-edf5-4c97-832c-cd3f3572bbe4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:56.198158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c290f9e-edf5-4c97-832c-cd3f3572bbe4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:56.203369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23bdff-599a-4472-8450-61a5c08a9700 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:56.203996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23bdff-599a-4472-8450-61a5c08a9700 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:56.204281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23bdff-599a-4472-8450-61a5c08a9700 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:56.204539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23bdff-599a-4472-8450-61a5c08a9700 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:56.212676 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23bdff-599a-4472-8450-61a5c08a9700 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:56.214259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23bdff-599a-4472-8450-61a5c08a9700 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:56.224079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 721/1428] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:57.093031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda025f-b629-4837-a3a2-146124a145ec None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:57.093602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda025f-b629-4837-a3a2-146124a145ec None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:57.093602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda025f-b629-4837-a3a2-146124a145ec None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:57.094992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:57.099899 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d6408-30cd-4230-b2e1-707f18c1635a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:57.100306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d6408-30cd-4230-b2e1-707f18c1635a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:57.100449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d6408-30cd-4230-b2e1-707f18c1635a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:57.118152 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7b7d6408-30cd-4230-b2e1-707f18c1635a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:57.119370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7b7d6408-30cd-4230-b2e1-707f18c1635a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:57.127755 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:57.285793 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda025f-b629-4837-a3a2-146124a145ec None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:57.301245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda025f-b629-4837-a3a2-146124a145ec None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:57.307213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f20f1-6355-423e-ba47-16a7fb1a5672 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:57.307864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f20f1-6355-423e-ba47-16a7fb1a5672 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:57.308144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f20f1-6355-423e-ba47-16a7fb1a5672 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:57.308414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f20f1-6355-423e-ba47-16a7fb1a5672 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:57.317750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f20f1-6355-423e-ba47-16a7fb1a5672 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:57.319394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f20f1-6355-423e-ba47-16a7fb1a5672 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:57.329279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 709/1431] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:58.191620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946609d-e976-4b99-bdfa-1b5423a76471 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:58.192198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946609d-e976-4b99-bdfa-1b5423a76471 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:58.192198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946609d-e976-4b99-bdfa-1b5423a76471 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:58.193518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:58.201593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fa916-ab8f-4118-9838-0a0a42762e88 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:58.202017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fa916-ab8f-4118-9838-0a0a42762e88 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:58.202139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3fa916-ab8f-4118-9838-0a0a42762e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:58.222831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0f3fa916-ab8f-4118-9838-0a0a42762e88 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:58.224026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0f3fa916-ab8f-4118-9838-0a0a42762e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:58.231810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:58.383457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946609d-e976-4b99-bdfa-1b5423a76471 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:58.398149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946609d-e976-4b99-bdfa-1b5423a76471 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:58.403435 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731ca4f-bd7e-4505-87c0-555be4c7afb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:58.404104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ca4f-bd7e-4505-87c0-555be4c7afb1 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:58.404386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ca4f-bd7e-4505-87c0-555be4c7afb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:58.404657 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ca4f-bd7e-4505-87c0-555be4c7afb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:58.413120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731ca4f-bd7e-4505-87c0-555be4c7afb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:58.414739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c731ca4f-bd7e-4505-87c0-555be4c7afb1 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:58.424032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 724/1434] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:19:59.270049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e53d6-90ad-4fbe-851b-d599c1214529 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:59.270521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e53d6-90ad-4fbe-851b-d599c1214529 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:59.270666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e53d6-90ad-4fbe-851b-d599c1214529 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:59.271980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:19:59.276796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705d64e-fa5a-41e5-8214-0814b1d470a1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:59.277233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705d64e-fa5a-41e5-8214-0814b1d470a1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:59.277365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705d64e-fa5a-41e5-8214-0814b1d470a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:59.296869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7705d64e-fa5a-41e5-8214-0814b1d470a1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:19:59.298073 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7705d64e-fa5a-41e5-8214-0814b1d470a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:19:59.305807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:19:59.454751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e53d6-90ad-4fbe-851b-d599c1214529 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:19:59.469532 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e53d6-90ad-4fbe-851b-d599c1214529 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:19:59.474878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b4a30-18b4-4691-9e50-5753194f0ac8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:19:59.475508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b4a30-18b4-4691-9e50-5753194f0ac8 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:19:59.475789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b4a30-18b4-4691-9e50-5753194f0ac8 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:19:59.476067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b4a30-18b4-4691-9e50-5753194f0ac8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:19:59.484958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b4a30-18b4-4691-9e50-5753194f0ac8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:19:59.486598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b4a30-18b4-4691-9e50-5753194f0ac8 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:19:59.496265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 712/1437] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:00.354093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed4d2-df7d-4f7b-846a-69e8977c08ff None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:00.354690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed4d2-df7d-4f7b-846a-69e8977c08ff None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:00.354690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed4d2-df7d-4f7b-846a-69e8977c08ff None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:00.356139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:00.361630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c863-d636-4f18-a9df-6df254256f3f None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:00.362040 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c863-d636-4f18-a9df-6df254256f3f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:00.362175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4c863-d636-4f18-a9df-6df254256f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:00.380447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e5d4c863-d636-4f18-a9df-6df254256f3f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:00.381628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e5d4c863-d636-4f18-a9df-6df254256f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:00.389793 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:00.540726 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed4d2-df7d-4f7b-846a-69e8977c08ff None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:00.556146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed4d2-df7d-4f7b-846a-69e8977c08ff None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:00.562776 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93815c4c-b50d-4598-b18d-0304293c0a27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:00.563414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93815c4c-b50d-4598-b18d-0304293c0a27 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:00.563762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93815c4c-b50d-4598-b18d-0304293c0a27 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:00.563970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93815c4c-b50d-4598-b18d-0304293c0a27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:00.572737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93815c4c-b50d-4598-b18d-0304293c0a27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:00.574548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93815c4c-b50d-4598-b18d-0304293c0a27 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:00.584184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 727/1440] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:02.796793 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe9445-7b1e-4ae9-bf3f-a3445304435a None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:02.797392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe9445-7b1e-4ae9-bf3f-a3445304435a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:02.797392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe9445-7b1e-4ae9-bf3f-a3445304435a None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:02.798736 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:02.804332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bf9c4-ea59-40ef-8eef-24d347812388 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:02.804782 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bf9c4-ea59-40ef-8eef-24d347812388 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:02.804895 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4bf9c4-ea59-40ef-8eef-24d347812388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:02.822795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-cf4bf9c4-ea59-40ef-8eef-24d347812388 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:02.824005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cf4bf9c4-ea59-40ef-8eef-24d347812388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:02.832402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:02.871530 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe9445-7b1e-4ae9-bf3f-a3445304435a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:02.883033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe9445-7b1e-4ae9-bf3f-a3445304435a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:02.886779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913819a7-b0dc-48ca-b5d2-2f6ec9a47dc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:02.887427 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913819a7-b0dc-48ca-b5d2-2f6ec9a47dc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:02.887703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913819a7-b0dc-48ca-b5d2-2f6ec9a47dc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:02.887969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913819a7-b0dc-48ca-b5d2-2f6ec9a47dc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:02.898165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913819a7-b0dc-48ca-b5d2-2f6ec9a47dc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:02.899636 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913819a7-b0dc-48ca-b5d2-2f6ec9a47dc5 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:02.909985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 715/1443] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:05.325735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca2be4-1542-4e90-bd80-d517b8af8818 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:05.326311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca2be4-1542-4e90-bd80-d517b8af8818 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:05.326311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca2be4-1542-4e90-bd80-d517b8af8818 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:05.327835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:05.333696 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9621d-671b-46f9-a3b6-d2fd0252ca55 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:05.334114 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9621d-671b-46f9-a3b6-d2fd0252ca55 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:05.334250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9621d-671b-46f9-a3b6-d2fd0252ca55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:05.351586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-1fe9621d-671b-46f9-a3b6-d2fd0252ca55 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:05.352762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1fe9621d-671b-46f9-a3b6-d2fd0252ca55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:05.360512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:05.398166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca2be4-1542-4e90-bd80-d517b8af8818 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:05.408940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca2be4-1542-4e90-bd80-d517b8af8818 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:05.412360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42ebd9-4d32-4abb-b5e8-e6efcd555ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:05.412985 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42ebd9-4d32-4abb-b5e8-e6efcd555ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:05.413268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42ebd9-4d32-4abb-b5e8-e6efcd555ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:05.413544 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42ebd9-4d32-4abb-b5e8-e6efcd555ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:05.422349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42ebd9-4d32-4abb-b5e8-e6efcd555ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:05.423807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42ebd9-4d32-4abb-b5e8-e6efcd555ed2 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:05.433326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 730/1446] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:08.014318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0ec8e-ae38-4d39-b241-e8e410f925e2 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:08.014958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0ec8e-ae38-4d39-b241-e8e410f925e2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:08.014958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0ec8e-ae38-4d39-b241-e8e410f925e2 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:08.016290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:08.022101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182663f4-bbd6-4e99-b7c6-8d68854c6117 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:08.022621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182663f4-bbd6-4e99-b7c6-8d68854c6117 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:08.022796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182663f4-bbd6-4e99-b7c6-8d68854c6117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:08.041738 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-182663f4-bbd6-4e99-b7c6-8d68854c6117 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:08.043033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-182663f4-bbd6-4e99-b7c6-8d68854c6117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:08.051697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:08.091571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0ec8e-ae38-4d39-b241-e8e410f925e2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:08.102415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0ec8e-ae38-4d39-b241-e8e410f925e2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:08.105957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f6b07-7715-4f1b-823c-afaaf9a82bab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:08.106638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f6b07-7715-4f1b-823c-afaaf9a82bab service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:08.106921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f6b07-7715-4f1b-823c-afaaf9a82bab service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:08.107684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f6b07-7715-4f1b-823c-afaaf9a82bab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:08.117362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f6b07-7715-4f1b-823c-afaaf9a82bab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:08.118840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f6b07-7715-4f1b-823c-afaaf9a82bab service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:08.128835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 718/1449] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:10.495680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805abfed-8b39-453c-bfb3-cdee395ddc10 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:10.496272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805abfed-8b39-453c-bfb3-cdee395ddc10 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:10.496272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805abfed-8b39-453c-bfb3-cdee395ddc10 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:10.497590 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:10.503505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a870c22-826d-4539-9725-0b438256f5b8 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:10.504249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a870c22-826d-4539-9725-0b438256f5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:10.504249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a870c22-826d-4539-9725-0b438256f5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:10.524886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2a870c22-826d-4539-9725-0b438256f5b8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:10.526089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2a870c22-826d-4539-9725-0b438256f5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:10.534979 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:20:10.573910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805abfed-8b39-453c-bfb3-cdee395ddc10 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:10.584977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805abfed-8b39-453c-bfb3-cdee395ddc10 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:10.589611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfee50-8a07-4af7-af30-7fc0a3ce6a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:10.590253 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfee50-8a07-4af7-af30-7fc0a3ce6a0e service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:10.590553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfee50-8a07-4af7-af30-7fc0a3ce6a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:10.590840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfee50-8a07-4af7-af30-7fc0a3ce6a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:10.600422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bfee50-8a07-4af7-af30-7fc0a3ce6a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:10.601895 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bfee50-8a07-4af7-af30-7fc0a3ce6a0e service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:10.612434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 733/1452] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:12.932331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181f2ff-f5c3-4f45-b76b-8de2207af79f None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:12.932888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181f2ff-f5c3-4f45-b76b-8de2207af79f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:12.932888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181f2ff-f5c3-4f45-b76b-8de2207af79f None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:12.934219 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:20:12.939732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706f03b-b1b5-49a9-b2c5-923243e4290f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:12.940187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706f03b-b1b5-49a9-b2c5-923243e4290f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:12.940314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706f03b-b1b5-49a9-b2c5-923243e4290f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:12.958023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5706f03b-b1b5-49a9-b2c5-923243e4290f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:12.959230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5706f03b-b1b5-49a9-b2c5-923243e4290f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:12.967348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:13.005381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181f2ff-f5c3-4f45-b76b-8de2207af79f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:13.016575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181f2ff-f5c3-4f45-b76b-8de2207af79f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:13.020118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b4b40-9e6b-4705-9d44-8a0781350e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:13.020748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4b40-9e6b-4705-9d44-8a0781350e93 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:13.021032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4b40-9e6b-4705-9d44-8a0781350e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:13.021307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b4b40-9e6b-4705-9d44-8a0781350e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:13.030906 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b4b40-9e6b-4705-9d44-8a0781350e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:13.032600 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b4b40-9e6b-4705-9d44-8a0781350e93 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:13.041861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 721/1455] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:15.337207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfe3c7-31c2-4a46-85ea-5468ad8df85a None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:15.337767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfe3c7-31c2-4a46-85ea-5468ad8df85a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:15.337767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfe3c7-31c2-4a46-85ea-5468ad8df85a None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:15.339149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:15.345225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65344b29-f884-49b8-af45-9ccc5c1f752a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:15.345654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65344b29-f884-49b8-af45-9ccc5c1f752a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:15.345758 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65344b29-f884-49b8-af45-9ccc5c1f752a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:15.351771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=70 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:20:15.364785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-65344b29-f884-49b8-af45-9ccc5c1f752a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:15.366000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-65344b29-f884-49b8-af45-9ccc5c1f752a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:15.373852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:15.415465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfe3c7-31c2-4a46-85ea-5468ad8df85a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:15.420892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=64 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:20:15.427411 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfe3c7-31c2-4a46-85ea-5468ad8df85a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:15.430896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4a9fa-ce6b-4ac4-ba19-ad74b707d5b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:15.431534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4a9fa-ce6b-4ac4-ba19-ad74b707d5b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:15.431819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4a9fa-ce6b-4ac4-ba19-ad74b707d5b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:15.432099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4a9fa-ce6b-4ac4-ba19-ad74b707d5b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:15.440936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4a9fa-ce6b-4ac4-ba19-ad74b707d5b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:15.442373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4a9fa-ce6b-4ac4-ba19-ad74b707d5b1 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:15.452599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 736/1458] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:17.768131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afea3b-3e0c-495b-b2aa-20a3c470a188 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:17.768691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afea3b-3e0c-495b-b2aa-20a3c470a188 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:17.768691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33afea3b-3e0c-495b-b2aa-20a3c470a188 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:17.770065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:17.775499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278e43c-257d-475d-8239-be7b4715c742 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:17.775907 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278e43c-257d-475d-8239-be7b4715c742 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:17.776093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278e43c-257d-475d-8239-be7b4715c742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:17.794310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1278e43c-257d-475d-8239-be7b4715c742 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:17.795541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1278e43c-257d-475d-8239-be7b4715c742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:17.803321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:17.840546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afea3b-3e0c-495b-b2aa-20a3c470a188 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:17.851687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33afea3b-3e0c-495b-b2aa-20a3c470a188 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:17.855345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33120353-aad1-40a3-b841-7db61c1acc8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:17.855983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33120353-aad1-40a3-b841-7db61c1acc8f service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:17.856265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33120353-aad1-40a3-b841-7db61c1acc8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:17.856538 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33120353-aad1-40a3-b841-7db61c1acc8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:17.865767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33120353-aad1-40a3-b841-7db61c1acc8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:17.867291 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33120353-aad1-40a3-b841-7db61c1acc8f service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:17.877123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 724/1461] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:20.207012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb701a76-5d24-4ac6-871e-eeef9850c2a9 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:20.207703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb701a76-5d24-4ac6-871e-eeef9850c2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:20.207703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb701a76-5d24-4ac6-871e-eeef9850c2a9 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:20.208919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:20.214990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6678c-1b6a-4b1d-b30c-aff16a784901 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:20.215572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6678c-1b6a-4b1d-b30c-aff16a784901 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:20.215572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6678c-1b6a-4b1d-b30c-aff16a784901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:20.235097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-9ed6678c-1b6a-4b1d-b30c-aff16a784901 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:20.236294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9ed6678c-1b6a-4b1d-b30c-aff16a784901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:20.244118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:20.281855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb701a76-5d24-4ac6-871e-eeef9850c2a9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:20.292718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb701a76-5d24-4ac6-871e-eeef9850c2a9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:20.296832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210ca8b-8d6f-484e-8e00-c1322e85e035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:20.297463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ca8b-8d6f-484e-8e00-c1322e85e035 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:20.297745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ca8b-8d6f-484e-8e00-c1322e85e035 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:20.298019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ca8b-8d6f-484e-8e00-c1322e85e035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:20.306789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210ca8b-8d6f-484e-8e00-c1322e85e035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:20.308218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210ca8b-8d6f-484e-8e00-c1322e85e035 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:20.317444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 739/1464] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:22.600183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573a85-95bc-45bc-9ea8-069045329c91 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:22.600767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573a85-95bc-45bc-9ea8-069045329c91 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:22.600767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573a85-95bc-45bc-9ea8-069045329c91 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:22.602136 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:22.607802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720fceb-0434-40ec-9062-ab7046b337d8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:22.608249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720fceb-0434-40ec-9062-ab7046b337d8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:22.608363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720fceb-0434-40ec-9062-ab7046b337d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:22.626634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e720fceb-0434-40ec-9062-ab7046b337d8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:22.627826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e720fceb-0434-40ec-9062-ab7046b337d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:22.639931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:22.681480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573a85-95bc-45bc-9ea8-069045329c91 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:22.693351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573a85-95bc-45bc-9ea8-069045329c91 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:22.696909 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423d1f6-9dc5-4d7d-b576-0d93343a4bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:22.697556 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423d1f6-9dc5-4d7d-b576-0d93343a4bbb service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:22.697832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423d1f6-9dc5-4d7d-b576-0d93343a4bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:22.698106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423d1f6-9dc5-4d7d-b576-0d93343a4bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:22.708019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423d1f6-9dc5-4d7d-b576-0d93343a4bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:22.709457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423d1f6-9dc5-4d7d-b576-0d93343a4bbb service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:22.719026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 727/1467] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:24.974200 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cfa4e-38c9-47c4-9716-734c44ed7339 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:24.974785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cfa4e-38c9-47c4-9716-734c44ed7339 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:24.974785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9cfa4e-38c9-47c4-9716-734c44ed7339 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:24.976048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:24.981380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642409-9568-44fb-9de5-c00a3ef42be1 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:24.982378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642409-9568-44fb-9de5-c00a3ef42be1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:24.982487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642409-9568-44fb-9de5-c00a3ef42be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:24.999923 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-3c642409-9568-44fb-9de5-c00a3ef42be1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:25.001106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3c642409-9568-44fb-9de5-c00a3ef42be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:25.008722 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:25.045354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9cfa4e-38c9-47c4-9716-734c44ed7339 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:25.055958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9cfa4e-38c9-47c4-9716-734c44ed7339 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:25.059513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36beb8-a037-49be-ba41-6a9b8a68f860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:25.060142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36beb8-a037-49be-ba41-6a9b8a68f860 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:25.060424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36beb8-a037-49be-ba41-6a9b8a68f860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:25.060700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36beb8-a037-49be-ba41-6a9b8a68f860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:25.069444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36beb8-a037-49be-ba41-6a9b8a68f860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:25.070922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36beb8-a037-49be-ba41-6a9b8a68f860 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:25.081333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 742/1470] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:20:34.994897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=20 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:20:35.076628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=23 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:20:57.201962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf583d56-eaca-46b0-8e67-3ba107d5bb3f None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:57.202623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf583d56-eaca-46b0-8e67-3ba107d5bb3f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:57.202623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf583d56-eaca-46b0-8e67-3ba107d5bb3f None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:57.205037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 729/1471] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:57.210518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a82cc9-bb39-4e53-9144-66c0b4a89946 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:57.211024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a82cc9-bb39-4e53-9144-66c0b4a89946 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:57.211136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a82cc9-bb39-4e53-9144-66c0b4a89946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:57.229480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-82a82cc9-bb39-4e53-9144-66c0b4a89946 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:57.230719 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-82a82cc9-bb39-4e53-9144-66c0b4a89946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:57.239610 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 743/1472] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:57.296634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e67b5c-1b91-43c4-8a00-b7a70a772f54 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:57.296926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e67b5c-1b91-43c4-8a00-b7a70a772f54 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:57.297215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e67b5c-1b91-43c4-8a00-b7a70a772f54 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:57.298597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 730/1473] 10.222.0.93 () {54 vars in 962 bytes} [Fri Jun 3 15:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:57.303669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f95f1-870b-47b2-afe9-e10f09a3abf8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:57.303957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f95f1-870b-47b2-afe9-e10f09a3abf8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:57.304234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f95f1-870b-47b2-afe9-e10f09a3abf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:57.323206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e69f95f1-870b-47b2-afe9-e10f09a3abf8 None None] MFA Rules not processed for user `13121a263c31433dbd4a5082b4bf3bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:57.324408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e69f95f1-870b-47b2-afe9-e10f09a3abf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:57.331776 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 744/1474] 10.222.0.93 () {58 vars in 1059 bytes} [Fri Jun 3 15:20:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:57.339191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e67b5c-1b91-43c4-8a00-b7a70a772f54 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:57.350510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e67b5c-1b91-43c4-8a00-b7a70a772f54 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:57.354017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab9e03-4ad2-4355-a29d-6fca7082d135 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13121a263c31433dbd4a5082b4bf3bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:57.354765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab9e03-4ad2-4355-a29d-6fca7082d135 service glance] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:57.354955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab9e03-4ad2-4355-a29d-6fca7082d135 service glance] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:57.355229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab9e03-4ad2-4355-a29d-6fca7082d135 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:57.365943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab9e03-4ad2-4355-a29d-6fca7082d135 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:57.366601 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab9e03-4ad2-4355-a29d-6fca7082d135 service glance] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:57.377309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 731/1475] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:20:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:20:59.759832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26310a-2bd7-42b4-aa34-aa8305c37232 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:59.765555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26310a-2bd7-42b4-aa34-aa8305c37232 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:59.765555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26310a-2bd7-42b4-aa34-aa8305c37232 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:59.765555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 745/1476] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:20:59.767941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaf9f6-8fe9-4179-a88f-50abaa11289e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:59.768437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaf9f6-8fe9-4179-a88f-50abaa11289e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:59.768545 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaf9f6-8fe9-4179-a88f-50abaa11289e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:59.787585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2ecaf9f6-8fe9-4179-a88f-50abaa11289e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:20:59.788784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2ecaf9f6-8fe9-4179-a88f-50abaa11289e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:20:59.798816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 732/1477] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:20:59.823090 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26310a-2bd7-42b4-aa34-aa8305c37232 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:20:59.837018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26310a-2bd7-42b4-aa34-aa8305c37232 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:20:59.841730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1918c0-7646-4b23-8263-0d90c3641fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:20:59.842381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1918c0-7646-4b23-8263-0d90c3641fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:20:59.842699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1918c0-7646-4b23-8263-0d90c3641fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:20:59.842974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1918c0-7646-4b23-8263-0d90c3641fe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:20:59.843708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1918c0-7646-4b23-8263-0d90c3641fe4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:20:59.845464 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-1c1918c0-7646-4b23-8263-0d90c3641fe4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:20:59.845464 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 03 15:20:59.845464 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: Jun 03 15:20:59.846647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1918c0-7646-4b23-8263-0d90c3641fe4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:20:59.869680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 746/1478] 10.222.0.93 () {56 vars in 1108 bytes} [Fri Jun 3 15:20:59 2022] GET /identity/v3/projects => generated 2553 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:21:02.151836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaed26-c6b5-4e69-b4f6-e721ddcb37af None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:02.152419 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaed26-c6b5-4e69-b4f6-e721ddcb37af None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:02.152419 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaed26-c6b5-4e69-b4f6-e721ddcb37af None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:02.153858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 733/1479] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:21:02.159680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9b651-5ef6-43e5-bf6b-02cbdd50cc97 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:02.160112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9b651-5ef6-43e5-bf6b-02cbdd50cc97 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:02.160260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9b651-5ef6-43e5-bf6b-02cbdd50cc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:02.183212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-60c9b651-5ef6-43e5-bf6b-02cbdd50cc97 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:21:02.184929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-60c9b651-5ef6-43e5-bf6b-02cbdd50cc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:21:02.192999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 747/1480] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:21:02.239025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaed26-c6b5-4e69-b4f6-e721ddcb37af None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:21:02.254619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaed26-c6b5-4e69-b4f6-e721ddcb37af None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:21:02.260193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7441ed6-0af4-430a-9bcb-4793ad0958c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:21:02.260863 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7441ed6-0af4-430a-9bcb-4793ad0958c8 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:02.261151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7441ed6-0af4-430a-9bcb-4793ad0958c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:02.261434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7441ed6-0af4-430a-9bcb-4793ad0958c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:02.271359 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7441ed6-0af4-430a-9bcb-4793ad0958c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:21:02.272994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7441ed6-0af4-430a-9bcb-4793ad0958c8 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:21:02.284101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 734/1481] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:21:04.700607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ec7af-6ba4-4c2e-8ceb-b3216b725ddb None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:04.701115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ec7af-6ba4-4c2e-8ceb-b3216b725ddb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:04.701214 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ec7af-6ba4-4c2e-8ceb-b3216b725ddb None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:04.702508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 748/1482] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:21:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:21:04.707817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af343-1479-43e6-a992-00c7f2958e2e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:04.708251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af343-1479-43e6-a992-00c7f2958e2e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:04.708382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044af343-1479-43e6-a992-00c7f2958e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:04.726074 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-044af343-1479-43e6-a992-00c7f2958e2e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:21:04.727323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-044af343-1479-43e6-a992-00c7f2958e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:21:04.735212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 735/1483] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:21:04.771744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ec7af-6ba4-4c2e-8ceb-b3216b725ddb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:21:04.782804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ec7af-6ba4-4c2e-8ceb-b3216b725ddb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:21:04.786204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea7989-4314-4b06-9e45-915421eda000 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:21:04.786870 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea7989-4314-4b06-9e45-915421eda000 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:04.787150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea7989-4314-4b06-9e45-915421eda000 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:04.787422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea7989-4314-4b06-9e45-915421eda000 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:04.797722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea7989-4314-4b06-9e45-915421eda000 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:21:04.799216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea7989-4314-4b06-9e45-915421eda000 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:21:04.808926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 749/1484] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:21:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:21:07.103055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bab38-93c2-43b8-ae41-25012e011a7a None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:07.103642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bab38-93c2-43b8-ae41-25012e011a7a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:07.103642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7bab38-93c2-43b8-ae41-25012e011a7a None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:07.104959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 736/1485] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:21:07.110847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41de8da-1c06-4016-9219-302756f827e0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:07.111281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41de8da-1c06-4016-9219-302756f827e0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:07.111394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41de8da-1c06-4016-9219-302756f827e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:07.129383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b41de8da-1c06-4016-9219-302756f827e0 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:21:07.130614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b41de8da-1c06-4016-9219-302756f827e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:21:07.138289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 750/1486] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:21:07.175603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bab38-93c2-43b8-ae41-25012e011a7a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:21:07.186820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7bab38-93c2-43b8-ae41-25012e011a7a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:21:07.190453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe418a-46ce-4ea1-8024-dbd8723b1a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:21:07.191110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe418a-46ce-4ea1-8024-dbd8723b1a39 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:07.191387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe418a-46ce-4ea1-8024-dbd8723b1a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:07.191659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe418a-46ce-4ea1-8024-dbd8723b1a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:07.200771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe418a-46ce-4ea1-8024-dbd8723b1a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:21:07.202197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe418a-46ce-4ea1-8024-dbd8723b1a39 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:21:07.211468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 737/1487] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:21:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:21:09.601428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faaf23-9631-4b91-afdf-0d914f84d03e None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:09.602023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faaf23-9631-4b91-afdf-0d914f84d03e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:09.602023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3faaf23-9631-4b91-afdf-0d914f84d03e None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:09.603284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 751/1488] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:21:09.608514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49666d32-0382-4025-97d6-9c399642cfe4 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:09.608966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49666d32-0382-4025-97d6-9c399642cfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:09.609093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49666d32-0382-4025-97d6-9c399642cfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:09.627210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-49666d32-0382-4025-97d6-9c399642cfe4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:21:09.628397 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-49666d32-0382-4025-97d6-9c399642cfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:21:09.636080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 738/1489] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:21:09.676343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3faaf23-9631-4b91-afdf-0d914f84d03e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:21:09.687353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3faaf23-9631-4b91-afdf-0d914f84d03e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:21:09.690881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7c243-aea7-41cf-b9f5-9de81efeab10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:21:09.691509 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7c243-aea7-41cf-b9f5-9de81efeab10 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:09.691787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7c243-aea7-41cf-b9f5-9de81efeab10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:09.692059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7c243-aea7-41cf-b9f5-9de81efeab10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:09.701866 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7c243-aea7-41cf-b9f5-9de81efeab10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:21:09.703356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7c243-aea7-41cf-b9f5-9de81efeab10 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:21:09.712872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 752/1490] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:21:12.264077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa07a-ddc2-4f84-bbc8-70483e3cf46d None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:12.264650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa07a-ddc2-4f84-bbc8-70483e3cf46d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:12.264650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa07a-ddc2-4f84-bbc8-70483e3cf46d None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:12.266018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 739/1491] 10.222.0.93 () {54 vars in 926 bytes} [Fri Jun 3 15:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:21:12.271118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56afa0-1b75-4203-88bd-1d49288f0476 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:12.271496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56afa0-1b75-4203-88bd-1d49288f0476 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:12.271679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56afa0-1b75-4203-88bd-1d49288f0476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:12.289673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3f56afa0-1b75-4203-88bd-1d49288f0476 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:21:12.290882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3f56afa0-1b75-4203-88bd-1d49288f0476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:21:12.298442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 753/1492] 10.222.0.93 () {58 vars in 1023 bytes} [Fri Jun 3 15:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:21:12.336353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aa07a-ddc2-4f84-bbc8-70483e3cf46d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:21:12.347879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aa07a-ddc2-4f84-bbc8-70483e3cf46d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:21:12.351399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc7971-beb9-481c-97f8-db8c0c563ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:21:12.352028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc7971-beb9-481c-97f8-db8c0c563ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:21:12.352376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc7971-beb9-481c-97f8-db8c0c563ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:21:12.352676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc7971-beb9-481c-97f8-db8c0c563ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:21:12.361461 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc7971-beb9-481c-97f8-db8c0c563ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:21:12.362920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc7971-beb9-481c-97f8-db8c0c563ab6 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:21:12.372143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 740/1493] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:21:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:21:22.284722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=33 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:21:22.368070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=33 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:22:46.393219 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028c273-3a4e-4b94-bc8c-7b55b6ecea2d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:46.393863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028c273-3a4e-4b94-bc8c-7b55b6ecea2d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:46.393863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028c273-3a4e-4b94-bc8c-7b55b6ecea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:46.411592 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5028c273-3a4e-4b94-bc8c-7b55b6ecea2d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:22:46.412893 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5028c273-3a4e-4b94-bc8c-7b55b6ecea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:22:46.421529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 754/1494] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:22:46.429574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa7455-41ac-460b-a5f0-2a8da1f0ed03 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:46.430086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa7455-41ac-460b-a5f0-2a8da1f0ed03 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:46.430322 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa7455-41ac-460b-a5f0-2a8da1f0ed03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:46.460735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-8daa7455-41ac-460b-a5f0-2a8da1f0ed03 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:22:46.461971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8daa7455-41ac-460b-a5f0-2a8da1f0ed03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:22:46.470229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 741/1495] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:22:46.475359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028c273-3a4e-4b94-bc8c-7b55b6ecea2d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:22:46.486432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028c273-3a4e-4b94-bc8c-7b55b6ecea2d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:22:46.493215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03afae3-7d8f-4a7c-aae7-ab5e415d4fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:22:46.494085 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03afae3-7d8f-4a7c-aae7-ab5e415d4fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:46.494461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03afae3-7d8f-4a7c-aae7-ab5e415d4fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:46.495531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03afae3-7d8f-4a7c-aae7-ab5e415d4fc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:46.496400 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03afae3-7d8f-4a7c-aae7-ab5e415d4fc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:22:46.496819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03afae3-7d8f-4a7c-aae7-ab5e415d4fc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:22:46.497975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-c03afae3-7d8f-4a7c-aae7-ab5e415d4fc9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:22:46.497975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 03 15:22:46.497975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: Jun 03 15:22:46.499205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03afae3-7d8f-4a7c-aae7-ab5e415d4fc9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:22:46.512252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 755/1496] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:22:46.521132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa7455-41ac-460b-a5f0-2a8da1f0ed03 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:22:46.531012 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa7455-41ac-460b-a5f0-2a8da1f0ed03 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:22:46.535680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667415b-7f0c-471c-9675-986d8516cd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:22:46.536333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667415b-7f0c-471c-9675-986d8516cd6a admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:46.536620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667415b-7f0c-471c-9675-986d8516cd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:46.536898 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667415b-7f0c-471c-9675-986d8516cd6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:46.537587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667415b-7f0c-471c-9675-986d8516cd6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:22:46.539361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: WARNING py.warnings [None req-8667415b-7f0c-471c-9675-986d8516cd6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:22:46.539361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: warnings.warn(msg) Jun 03 15:22:46.539361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: Jun 03 15:22:46.541242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667415b-7f0c-471c-9675-986d8516cd6a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:22:46.578464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 742/1497] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:22:46 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:22:46.583157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03afae3-7d8f-4a7c-aae7-ab5e415d4fc9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:22:46.593556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03afae3-7d8f-4a7c-aae7-ab5e415d4fc9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:22:46.650308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4538b8-406c-4620-a362-e2d58ec10e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:22:46.650948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4538b8-406c-4620-a362-e2d58ec10e25 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:46.651232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4538b8-406c-4620-a362-e2d58ec10e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:46.651558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4538b8-406c-4620-a362-e2d58ec10e25 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:46.652269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4538b8-406c-4620-a362-e2d58ec10e25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:22:46.653292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-ee4538b8-406c-4620-a362-e2d58ec10e25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:22:46.653292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 03 15:22:46.653292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: Jun 03 15:22:46.654804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4538b8-406c-4620-a362-e2d58ec10e25 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:22:46.676768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 756/1498] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:22:46 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:22:46.680927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667415b-7f0c-471c-9675-986d8516cd6a admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:22:46.692447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667415b-7f0c-471c-9675-986d8516cd6a admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:22:46.697114 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dc950-bd35-4f21-b4e0-f0f943d3fb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:22:46.697858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dc950-bd35-4f21-b4e0-f0f943d3fb6f admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:46.698173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dc950-bd35-4f21-b4e0-f0f943d3fb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:46.698375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3dc950-bd35-4f21-b4e0-f0f943d3fb6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:46.699133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3dc950-bd35-4f21-b4e0-f0f943d3fb6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:22:46.700145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: WARNING py.warnings [None req-bf3dc950-bd35-4f21-b4e0-f0f943d3fb6f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:22:46.700145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: warnings.warn(msg) Jun 03 15:22:46.700145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: Jun 03 15:22:46.701039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3dc950-bd35-4f21-b4e0-f0f943d3fb6f admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:22:46.713950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 743/1499] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:22:46.718211 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4538b8-406c-4620-a362-e2d58ec10e25 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:22:46.730301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4538b8-406c-4620-a362-e2d58ec10e25 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:22:46.734936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad46e8-0e3c-4d08-882e-9a32af84d533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:22:46.735594 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad46e8-0e3c-4d08-882e-9a32af84d533 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:46.735880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad46e8-0e3c-4d08-882e-9a32af84d533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:46.736160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad46e8-0e3c-4d08-882e-9a32af84d533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:46.736879 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad46e8-0e3c-4d08-882e-9a32af84d533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:22:46.737929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-93ad46e8-0e3c-4d08-882e-9a32af84d533 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:22:46.737929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 03 15:22:46.737929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: Jun 03 15:22:46.738850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad46e8-0e3c-4d08-882e-9a32af84d533 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:22:46.752819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 757/1500] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:22:46.756967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dc950-bd35-4f21-b4e0-f0f943d3fb6f admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:22:46.766946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3dc950-bd35-4f21-b4e0-f0f943d3fb6f admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:22:46.772294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb0697-e095-456d-86c8-c41a7b4b6b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:22:46.772584 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0697-e095-456d-86c8-c41a7b4b6b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:46.772873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0697-e095-456d-86c8-c41a7b4b6b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:46.773149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0697-e095-456d-86c8-c41a7b4b6b1c admin admin] PATH_INFO: `/v3/projects/4ce6941c496f4939b0f36d780027887c/users/6260833e34724df796f1b4942c8d157f/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:46.789333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb0697-e095-456d-86c8-c41a7b4b6b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ce6941c496f4939b0f36d780027887c, user_id=6260833e34724df796f1b4942c8d157f, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:22:46.790356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: WARNING py.warnings [None req-05cb0697-e095-456d-86c8-c41a7b4b6b1c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:22:46.790356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: warnings.warn(msg) Jun 03 15:22:46.790356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: Jun 03 15:22:46.792852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb0697-e095-456d-86c8-c41a7b4b6b1c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:22:46.807127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 744/1501] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:22:46 2022] PUT /identity/v3/projects/4ce6941c496f4939b0f36d780027887c/users/6260833e34724df796f1b4942c8d157f/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:22:46.819817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251b2ee-d57b-4066-a939-cc7b0a9e2340 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:46.820104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251b2ee-d57b-4066-a939-cc7b0a9e2340 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:46.820381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251b2ee-d57b-4066-a939-cc7b0a9e2340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:46.846670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1251b2ee-d57b-4066-a939-cc7b0a9e2340 None None] MFA Rules not processed for user `6260833e34724df796f1b4942c8d157f`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:22:46.847907 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1251b2ee-d57b-4066-a939-cc7b0a9e2340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:22:46.915994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 758/1502] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:22:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:22:46.932083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ed30-286f-46e6-8bba-5c5d362fb57a None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:46.932371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ed30-286f-46e6-8bba-5c5d362fb57a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:46.932649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ed30-286f-46e6-8bba-5c5d362fb57a None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:46.934012 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 745/1503] 10.222.0.93 () {54 vars in 963 bytes} [Fri Jun 3 15:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:22:46.938958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682264d7-95ff-4706-b597-fad5877b7ac5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:46.939251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682264d7-95ff-4706-b597-fad5877b7ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:46.939532 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682264d7-95ff-4706-b597-fad5877b7ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:46.956974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-682264d7-95ff-4706-b597-fad5877b7ac5 None None] MFA Rules not processed for user `98aa51c4bc604b0ca8a6c69bc1455bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:22:46.958187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-682264d7-95ff-4706-b597-fad5877b7ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:22:47.003221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:22:47.013058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 759/1504] 10.222.0.93 () {58 vars in 1060 bytes} [Fri Jun 3 15:22:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:22:47.020309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ed30-286f-46e6-8bba-5c5d362fb57a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:22:47.031599 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ed30-286f-46e6-8bba-5c5d362fb57a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:22:47.035083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94b206-a358-4ee8-929e-3883c2974205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:22:47.035714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94b206-a358-4ee8-929e-3883c2974205 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:47.035996 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94b206-a358-4ee8-929e-3883c2974205 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:47.036272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94b206-a358-4ee8-929e-3883c2974205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:47.045063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94b206-a358-4ee8-929e-3883c2974205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:22:47.046490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94b206-a358-4ee8-929e-3883c2974205 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:22:47.055303 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 746/1505] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:22:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:22:47.097938 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8939b-4108-4a62-8c29-4679782e7d73 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:47.098233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8939b-4108-4a62-8c29-4679782e7d73 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:47.098516 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c8939b-4108-4a62-8c29-4679782e7d73 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:47.099824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 760/1506] 10.222.0.93 () {52 vars in 829 bytes} [Fri Jun 3 15:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:22:47.104745 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94b206-a358-4ee8-929e-3883c2974205 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:22:47.174498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94b206-a358-4ee8-929e-3883c2974205 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:22:47.179025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:22:47.179930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:47.180206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:47.180472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] PATH_INFO: `/v3/users/6260833e34724df796f1b4942c8d157f` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:47.181680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6260833e34724df796f1b4942c8d157f)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:22:47.182794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: WARNING py.warnings [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:22:47.182794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: warnings.warn(msg) Jun 03 15:22:47.182794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: Jun 03 15:22:47.183997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:22:47.248559 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6260833e34724df796f1b4942c8d157f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:22:47.267493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6260833e34724df796f1b4942c8d157f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:22:47.268302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6260833e34724df796f1b4942c8d157f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:22:47.281427 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6260833e34724df796f1b4942c8d157f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:22:47.282102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6260833e34724df796f1b4942c8d157f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:22:47.286092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:22:47.298036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 747/1507] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:22:47 2022] DELETE /identity/v3/users/6260833e34724df796f1b4942c8d157f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:22:47.302672 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c375c-d5df-4872-be37-e198b5b002fa None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:47.302967 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c375c-d5df-4872-be37-e198b5b002fa None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:47.303277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c375c-d5df-4872-be37-e198b5b002fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:47.322020 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-375c375c-d5df-4872-be37-e198b5b002fa None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:22:47.323795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-375c375c-d5df-4872-be37-e198b5b002fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:22:47.380392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 761/1508] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:22:47.391107 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:22:47.392537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:22:47.449674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f5b648-360c-4217-8ccc-191f48fe12d8 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:22:47.453123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a114af1-191c-4e1f-a1a9-8d24e9546241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:22:47.453765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a114af1-191c-4e1f-a1a9-8d24e9546241 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:47.454048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a114af1-191c-4e1f-a1a9-8d24e9546241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:47.454324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a114af1-191c-4e1f-a1a9-8d24e9546241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:47.463140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a114af1-191c-4e1f-a1a9-8d24e9546241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:22:47.465191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a114af1-191c-4e1f-a1a9-8d24e9546241 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:22:47.476173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 748/1509] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:22:47.805607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c375c-d5df-4872-be37-e198b5b002fa None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:22:47.807772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-375c375c-d5df-4872-be37-e198b5b002fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:22:47.820711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c375c-d5df-4872-be37-e198b5b002fa None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:22:47.827265 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:22:47.828132 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:22:47.828417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:22:47.828810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] PATH_INFO: `/v3/projects/4ce6941c496f4939b0f36d780027887c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:22:47.830562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ce6941c496f4939b0f36d780027887c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:22:47.832351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:22:47.832351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 03 15:22:47.832351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: Jun 03 15:22:47.833572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:22:47.899317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ce6941c496f4939b0f36d780027887c', 'request_id': 'req-8199b1ac-6d36-4208-bd10-0a24a167a3b3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:22:47.905918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ce6941c496f4939b0f36d780027887c', 'request_id': 'req-8199b1ac-6d36-4208-bd10-0a24a167a3b3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:22:47.917715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ce6941c496f4939b0f36d780027887c', 'request_id': 'req-8199b1ac-6d36-4208-bd10-0a24a167a3b3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:22:47.937635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] The token cache is being invalidate because project 4ce6941c496f4939b0f36d780027887c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:22:47.937925 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:22:47.938711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8199b1ac-6d36-4208-bd10-0a24a167a3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:22:47.940251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 762/1510] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:22:47 2022] DELETE /identity/v3/projects/4ce6941c496f4939b0f36d780027887c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:22:57.470329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:22:57.934710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:24:30.448773 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba488a4-8841-428d-9268-aa13078615a2 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:30.449376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba488a4-8841-428d-9268-aa13078615a2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:30.449376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba488a4-8841-428d-9268-aa13078615a2 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:30.450772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:24:30.455640 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258718e3-f09b-4763-8f76-c0b2d2c8b4de None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:30.456101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258718e3-f09b-4763-8f76-c0b2d2c8b4de None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:30.456210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258718e3-f09b-4763-8f76-c0b2d2c8b4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:30.473682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-258718e3-f09b-4763-8f76-c0b2d2c8b4de None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:24:30.474900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-258718e3-f09b-4763-8f76-c0b2d2c8b4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:30.536137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:24:30.707064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba488a4-8841-428d-9268-aa13078615a2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:24:30.708749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-4ba488a4-8841-428d-9268-aa13078615a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:30.783640 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba488a4-8841-428d-9268-aa13078615a2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:24:30.789021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40586d2-8d2a-441f-8c56-91918b973a4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:24:30.789669 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40586d2-8d2a-441f-8c56-91918b973a4d service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:30.790699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40586d2-8d2a-441f-8c56-91918b973a4d service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:30.790982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40586d2-8d2a-441f-8c56-91918b973a4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:30.799573 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40586d2-8d2a-441f-8c56-91918b973a4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:24:30.801158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40586d2-8d2a-441f-8c56-91918b973a4d service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:24:30.810540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 750/1513] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:24:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:24:31.706646 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac717bc-5c17-427e-aebb-011f3abec5cf None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:31.707447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac717bc-5c17-427e-aebb-011f3abec5cf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:31.707447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac717bc-5c17-427e-aebb-011f3abec5cf None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:31.708607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:24:31.713652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cf00a-ddcc-4417-82f2-f69c7df22c0e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:31.714097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cf00a-ddcc-4417-82f2-f69c7df22c0e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:31.714195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cf00a-ddcc-4417-82f2-f69c7df22c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:31.731832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-1d8cf00a-ddcc-4417-82f2-f69c7df22c0e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:24:31.733041 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1d8cf00a-ddcc-4417-82f2-f69c7df22c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:31.740750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:24:31.895466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac717bc-5c17-427e-aebb-011f3abec5cf None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:24:31.910342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac717bc-5c17-427e-aebb-011f3abec5cf None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:24:31.916120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a434bc54-dba6-4ebe-8ad5-5d0ea5c7fc77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:24:31.916754 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434bc54-dba6-4ebe-8ad5-5d0ea5c7fc77 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:31.917033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434bc54-dba6-4ebe-8ad5-5d0ea5c7fc77 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:31.917306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a434bc54-dba6-4ebe-8ad5-5d0ea5c7fc77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:31.925552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434bc54-dba6-4ebe-8ad5-5d0ea5c7fc77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:24:31.927117 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a434bc54-dba6-4ebe-8ad5-5d0ea5c7fc77 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:24:31.936916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 765/1516] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:24:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:24:32.196226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126909e-1415-4b70-994e-f2f619949c9f None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:32.196434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126909e-1415-4b70-994e-f2f619949c9f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:32.196725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126909e-1415-4b70-994e-f2f619949c9f None None] PATH_INFO: `/v3` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:32.198045 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 752/1517] 10.222.0.95 () {54 vars in 925 bytes} [Fri Jun 3 15:24:32 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:24:32.204183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7951-e096-4883-bc57-ec930deb25fc None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:32.204475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7951-e096-4883-bc57-ec930deb25fc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:32.204756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b7951-e096-4883-bc57-ec930deb25fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:32.223059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-af3b7951-e096-4883-bc57-ec930deb25fc None None] MFA Rules not processed for user `50625c2b38ed41698ba6cccdb12730ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:24:32.224483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-af3b7951-e096-4883-bc57-ec930deb25fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:32.307408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 766/1518] 10.222.0.95 () {58 vars in 1015 bytes} [Fri Jun 3 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:24:36.523163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b0d7d-9300-4b1e-a41f-16071df42c50 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:36.523716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b0d7d-9300-4b1e-a41f-16071df42c50 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:36.523716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41b0d7d-9300-4b1e-a41f-16071df42c50 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:36.526138 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 753/1519] 10.222.0.93 () {54 vars in 960 bytes} [Fri Jun 3 15:24:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:24:36.531487 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c308780-f6de-4327-90d6-29f4d0ba9d40 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:36.531967 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c308780-f6de-4327-90d6-29f4d0ba9d40 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:36.532074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c308780-f6de-4327-90d6-29f4d0ba9d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:36.551808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1c308780-f6de-4327-90d6-29f4d0ba9d40 None None] MFA Rules not processed for user `50625c2b38ed41698ba6cccdb12730ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:24:36.552996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1c308780-f6de-4327-90d6-29f4d0ba9d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:36.561503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 767/1520] 10.222.0.93 () {58 vars in 1057 bytes} [Fri Jun 3 15:24:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:24:36.569162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b0d7d-9300-4b1e-a41f-16071df42c50 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:24:36.582333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41b0d7d-9300-4b1e-a41f-16071df42c50 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:24:36.587232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467afc4-5205-4278-9ad6-02a7696c2c17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:24:36.587861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467afc4-5205-4278-9ad6-02a7696c2c17 service placement] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:36.588659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467afc4-5205-4278-9ad6-02a7696c2c17 service placement] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:36.588944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467afc4-5205-4278-9ad6-02a7696c2c17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:36.597097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467afc4-5205-4278-9ad6-02a7696c2c17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:24:36.598718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467afc4-5205-4278-9ad6-02a7696c2c17 service placement] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:24:36.608914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 754/1521] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:24:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:24:43.184584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff62b2-e86b-4e74-9082-d77d71403123 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:43.185189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff62b2-e86b-4e74-9082-d77d71403123 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:43.185189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff62b2-e86b-4e74-9082-d77d71403123 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:43.186700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:24:43.191495 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d014222-e73b-48e6-9d04-8603ac189f82 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:43.191945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d014222-e73b-48e6-9d04-8603ac189f82 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:43.192053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d014222-e73b-48e6-9d04-8603ac189f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:43.210197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-7d014222-e73b-48e6-9d04-8603ac189f82 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:24:43.211449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7d014222-e73b-48e6-9d04-8603ac189f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:43.219827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:24:43.372113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff62b2-e86b-4e74-9082-d77d71403123 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:24:43.386781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff62b2-e86b-4e74-9082-d77d71403123 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:24:43.392019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f4a80-f620-4bb1-aa9d-6929d366a52b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:24:43.392678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4a80-f620-4bb1-aa9d-6929d366a52b service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:43.392958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4a80-f620-4bb1-aa9d-6929d366a52b service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:43.393231 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f4a80-f620-4bb1-aa9d-6929d366a52b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:43.401174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f4a80-f620-4bb1-aa9d-6929d366a52b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:24:43.403554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f4a80-f620-4bb1-aa9d-6929d366a52b service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:24:43.413014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 769/1524] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:24:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:24:44.279938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd3828-f6ad-4909-930d-329882c54cd2 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:44.280600 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd3828-f6ad-4909-930d-329882c54cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:44.280600 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd3828-f6ad-4909-930d-329882c54cd2 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:44.281884 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 3 15:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:24:44.286974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c6a22-e6b8-4bfe-8e49-8e7973725d6c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:44.287415 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c6a22-e6b8-4bfe-8e49-8e7973725d6c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:44.287522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c6a22-e6b8-4bfe-8e49-8e7973725d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:44.304994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c24c6a22-e6b8-4bfe-8e49-8e7973725d6c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:24:44.306217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c24c6a22-e6b8-4bfe-8e49-8e7973725d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:44.314012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 3 15:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:24:44.465559 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd3828-f6ad-4909-930d-329882c54cd2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:24:44.479945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd3828-f6ad-4909-930d-329882c54cd2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:24:44.485329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24063191-5bb5-4ff0-9719-141e97e8f7f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:24:44.486102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24063191-5bb5-4ff0-9719-141e97e8f7f0 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:44.486384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24063191-5bb5-4ff0-9719-141e97e8f7f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:44.486692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24063191-5bb5-4ff0-9719-141e97e8f7f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:44.494848 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24063191-5bb5-4ff0-9719-141e97e8f7f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:24:44.496943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24063191-5bb5-4ff0-9719-141e97e8f7f0 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:24:44.506625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 757/1527] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:24:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:24:48.838187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4ec86-66ba-4315-bd02-83d917aff057 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:48.838852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4ec86-66ba-4315-bd02-83d917aff057 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:48.838852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4ec86-66ba-4315-bd02-83d917aff057 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:48.840708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:24:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:24:48.846176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59500fe-81af-4d25-a9d3-558a5399f8f6 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:48.846675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59500fe-81af-4d25-a9d3-558a5399f8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:48.846786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59500fe-81af-4d25-a9d3-558a5399f8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:48.866761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e59500fe-81af-4d25-a9d3-558a5399f8f6 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:24:48.867970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e59500fe-81af-4d25-a9d3-558a5399f8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:48.875819 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:24:48.914317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4ec86-66ba-4315-bd02-83d917aff057 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:24:48.916041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-50d4ec86-66ba-4315-bd02-83d917aff057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:48.974381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4ec86-66ba-4315-bd02-83d917aff057 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:24:48.977854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33763e69-5a27-4545-ace0-11a3149649ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:24:48.978490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33763e69-5a27-4545-ace0-11a3149649ac service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:48.978811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33763e69-5a27-4545-ace0-11a3149649ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:48.979088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33763e69-5a27-4545-ace0-11a3149649ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:48.988342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33763e69-5a27-4545-ace0-11a3149649ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:24:48.989787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33763e69-5a27-4545-ace0-11a3149649ac service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:24:48.994722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:24:49.000304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 772/1530] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:24:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:24:51.234251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968f566-272e-44e7-8f61-ea56763ea176 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:51.234873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968f566-272e-44e7-8f61-ea56763ea176 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:51.234873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968f566-272e-44e7-8f61-ea56763ea176 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:51.236180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:24:51.241428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af75baa-eb61-4876-8852-bd47ea4c6773 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:51.241865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af75baa-eb61-4876-8852-bd47ea4c6773 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:51.241982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af75baa-eb61-4876-8852-bd47ea4c6773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:51.261481 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3af75baa-eb61-4876-8852-bd47ea4c6773 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:24:51.262710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3af75baa-eb61-4876-8852-bd47ea4c6773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:51.270506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:24:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:24:51.311597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968f566-272e-44e7-8f61-ea56763ea176 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:24:51.322392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968f566-272e-44e7-8f61-ea56763ea176 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:24:51.325893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0de04e-a313-4867-a4a4-89733c3bf6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:24:51.326547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0de04e-a313-4867-a4a4-89733c3bf6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:51.326830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0de04e-a313-4867-a4a4-89733c3bf6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:51.327100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0de04e-a313-4867-a4a4-89733c3bf6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:51.335962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0de04e-a313-4867-a4a4-89733c3bf6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:24:51.337390 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0de04e-a313-4867-a4a4-89733c3bf6b0 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:24:51.347176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 760/1533] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:24:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:24:55.146989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d964f4-31a0-4dac-8672-c08811bde880 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:55.147565 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d964f4-31a0-4dac-8672-c08811bde880 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:55.147667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d964f4-31a0-4dac-8672-c08811bde880 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:55.149050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:24:55.154735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98242239-2595-43b8-8519-7a76c2395673 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:55.155182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98242239-2595-43b8-8519-7a76c2395673 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:55.155289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98242239-2595-43b8-8519-7a76c2395673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:55.172814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-98242239-2595-43b8-8519-7a76c2395673 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:24:55.174043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-98242239-2595-43b8-8519-7a76c2395673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:55.182340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:24:55.457555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d964f4-31a0-4dac-8672-c08811bde880 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:24:55.459334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-88d964f4-31a0-4dac-8672-c08811bde880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:55.516744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d964f4-31a0-4dac-8672-c08811bde880 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:24:55.520708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dd7d6-270a-4941-a32c-b1a3e8e0983a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:24:55.521349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dd7d6-270a-4941-a32c-b1a3e8e0983a service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:55.521635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dd7d6-270a-4941-a32c-b1a3e8e0983a service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:55.521912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dd7d6-270a-4941-a32c-b1a3e8e0983a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:55.531741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dd7d6-270a-4941-a32c-b1a3e8e0983a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:24:55.533197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dd7d6-270a-4941-a32c-b1a3e8e0983a service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:24:55.542484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 775/1536] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:24:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:24:57.819117 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad012a-53c6-4044-ae19-b4cd4e68e9b1 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:57.819661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad012a-53c6-4044-ae19-b4cd4e68e9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:57.819754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad012a-53c6-4044-ae19-b4cd4e68e9b1 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:57.821222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 3 15:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:24:57.827025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aa4ad-d79a-4043-bb03-336b4f5f9112 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:57.827482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aa4ad-d79a-4043-bb03-336b4f5f9112 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:57.827600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9aa4ad-d79a-4043-bb03-336b4f5f9112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:57.846008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bd9aa4ad-d79a-4043-bb03-336b4f5f9112 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:24:57.847227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bd9aa4ad-d79a-4043-bb03-336b4f5f9112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:57.854815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 3 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:24:58.124928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad012a-53c6-4044-ae19-b4cd4e68e9b1 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:24:58.126676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5cad012a-53c6-4044-ae19-b4cd4e68e9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:24:58.134084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=65 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:24:58.140504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad012a-53c6-4044-ae19-b4cd4e68e9b1 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:24:58.144159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d8256-fcdb-4c8b-bfc8-8892d1ce632b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:24:58.144804 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d8256-fcdb-4c8b-bfc8-8892d1ce632b service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:24:58.145084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d8256-fcdb-4c8b-bfc8-8892d1ce632b service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:24:58.145360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d8256-fcdb-4c8b-bfc8-8892d1ce632b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:24:58.156154 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d8256-fcdb-4c8b-bfc8-8892d1ce632b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:24:58.157590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d8256-fcdb-4c8b-bfc8-8892d1ce632b service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:24:58.167384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 763/1539] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:25:07.840407 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=32 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:25:08.162983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:30.634525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7ecc6-55ad-4b79-9f1d-9a3a6ec3d8eb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:30.635180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7ecc6-55ad-4b79-9f1d-9a3a6ec3d8eb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:30.635180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc7ecc6-55ad-4b79-9f1d-9a3a6ec3d8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:30.672559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5dc7ecc6-55ad-4b79-9f1d-9a3a6ec3d8eb None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:30.673791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5dc7ecc6-55ad-4b79-9f1d-9a3a6ec3d8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:30.699277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 777/1540] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:30.710342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3e569-ad7b-40b5-9a3a-1c8a0dd5bd16 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:30.710986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3e569-ad7b-40b5-9a3a-1c8a0dd5bd16 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:30.710986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3e569-ad7b-40b5-9a3a-1c8a0dd5bd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:30.740281 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-20b3e569-ad7b-40b5-9a3a-1c8a0dd5bd16 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:30.741535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-20b3e569-ad7b-40b5-9a3a-1c8a0dd5bd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:30.755229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 764/1541] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:30.759783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7ecc6-55ad-4b79-9f1d-9a3a6ec3d8eb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:30.776281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc7ecc6-55ad-4b79-9f1d-9a3a6ec3d8eb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:30.785144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cc5a2-f828-4910-80a9-23cdbeb649da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:30.785818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cc5a2-f828-4910-80a9-23cdbeb649da admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:30.786101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cc5a2-f828-4910-80a9-23cdbeb649da admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:30.786384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6cc5a2-f828-4910-80a9-23cdbeb649da admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:30.787150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6cc5a2-f828-4910-80a9-23cdbeb649da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:30.787457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6cc5a2-f828-4910-80a9-23cdbeb649da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:30.789081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6cc5a2-f828-4910-80a9-23cdbeb649da admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:30.809040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 778/1542] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:26:30.817424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3e569-ad7b-40b5-9a3a-1c8a0dd5bd16 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:30.827762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3e569-ad7b-40b5-9a3a-1c8a0dd5bd16 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:30.840858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb90bb-2919-490e-ac31-9b27d7fc88a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:30.841540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb90bb-2919-490e-ac31-9b27d7fc88a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:30.841818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb90bb-2919-490e-ac31-9b27d7fc88a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:30.842089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb90bb-2919-490e-ac31-9b27d7fc88a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:30.842929 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb90bb-2919-490e-ac31-9b27d7fc88a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:30.844835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb90bb-2919-490e-ac31-9b27d7fc88a9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:30.912683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 765/1543] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:30 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:30.917107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cc5a2-f828-4910-80a9-23cdbeb649da admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:30.930093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6cc5a2-f828-4910-80a9-23cdbeb649da admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:31.003780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363765f0-39bc-4a60-b482-c45a7e2bad04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:31.004487 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363765f0-39bc-4a60-b482-c45a7e2bad04 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.004771 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363765f0-39bc-4a60-b482-c45a7e2bad04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.005046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363765f0-39bc-4a60-b482-c45a7e2bad04 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.005760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363765f0-39bc-4a60-b482-c45a7e2bad04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:31.007760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363765f0-39bc-4a60-b482-c45a7e2bad04 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:31.061521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 779/1544] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:30 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:31.065854 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb90bb-2919-490e-ac31-9b27d7fc88a9 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:31.085428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb90bb-2919-490e-ac31-9b27d7fc88a9 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:31.096683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eab912-0c6d-4d2b-a8b3-a96264a877d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:31.097379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eab912-0c6d-4d2b-a8b3-a96264a877d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.097660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eab912-0c6d-4d2b-a8b3-a96264a877d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.097934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eab912-0c6d-4d2b-a8b3-a96264a877d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.098692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eab912-0c6d-4d2b-a8b3-a96264a877d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:31.100308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eab912-0c6d-4d2b-a8b3-a96264a877d2 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:31.116784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 766/1545] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:31.121498 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363765f0-39bc-4a60-b482-c45a7e2bad04 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:31.132272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363765f0-39bc-4a60-b482-c45a7e2bad04 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:31.146379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ab40b-5c9f-4f03-baee-09620ba77cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:31.147291 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab40b-5c9f-4f03-baee-09620ba77cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.147730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab40b-5c9f-4f03-baee-09620ba77cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.148064 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab40b-5c9f-4f03-baee-09620ba77cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.148873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ab40b-5c9f-4f03-baee-09620ba77cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:31.151107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ab40b-5c9f-4f03-baee-09620ba77cc7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:31.168673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 780/1546] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:31.175933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eab912-0c6d-4d2b-a8b3-a96264a877d2 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:31.193063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eab912-0c6d-4d2b-a8b3-a96264a877d2 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:31.200001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa16cb-18f3-4698-bef7-e68f43628fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:31.201032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa16cb-18f3-4698-bef7-e68f43628fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.201397 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa16cb-18f3-4698-bef7-e68f43628fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.201665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa16cb-18f3-4698-bef7-e68f43628fa8 admin admin] PATH_INFO: `/v3/projects/16edd3f309d54728882acc6d23f8861e/users/8632496bdc1b4d06a74caf483c8b0a42/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.216451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044ffef-993a-4e33-becc-16d5a62f89b4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.216792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044ffef-993a-4e33-becc-16d5a62f89b4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.216991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044ffef-993a-4e33-becc-16d5a62f89b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.248537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa16cb-18f3-4698-bef7-e68f43628fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16edd3f309d54728882acc6d23f8861e, user_id=8632496bdc1b4d06a74caf483c8b0a42, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:31.251293 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa16cb-18f3-4698-bef7-e68f43628fa8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:31.252772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1044ffef-993a-4e33-becc-16d5a62f89b4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.254028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1044ffef-993a-4e33-becc-16d5a62f89b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.271711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 781/1547] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.286084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e781eb-1ea0-488e-a6c3-48f4fa31af73 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.286421 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e781eb-1ea0-488e-a6c3-48f4fa31af73 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.286668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e781eb-1ea0-488e-a6c3-48f4fa31af73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.294417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 767/1548] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:31 2022] PUT /identity/v3/projects/16edd3f309d54728882acc6d23f8861e/users/8632496bdc1b4d06a74caf483c8b0a42/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:31.296148 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b2214-973a-414c-9df2-e74b10f03fff None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.296447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b2214-973a-414c-9df2-e74b10f03fff None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.296728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b2214-973a-414c-9df2-e74b10f03fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.343065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e2e781eb-1ea0-488e-a6c3-48f4fa31af73 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.344277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e2e781eb-1ea0-488e-a6c3-48f4fa31af73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.347879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e93b2214-973a-414c-9df2-e74b10f03fff None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.349139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e93b2214-973a-414c-9df2-e74b10f03fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.477377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 768/1549] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:26:31.492914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b2214-973a-414c-9df2-e74b10f03fff None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:31.560844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 782/1550] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.562466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81d1f2-939f-4a80-ad18-fadc5d90f19e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.566555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81d1f2-939f-4a80-ad18-fadc5d90f19e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.566872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81d1f2-939f-4a80-ad18-fadc5d90f19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.607066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bb81d1f2-939f-4a80-ad18-fadc5d90f19e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.607709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b2214-973a-414c-9df2-e74b10f03fff None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:31.609329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bb81d1f2-939f-4a80-ad18-fadc5d90f19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.614873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565a79f-1a1d-45ca-94be-32de7c6681f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:31.614873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565a79f-1a1d-45ca-94be-32de7c6681f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.614873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565a79f-1a1d-45ca-94be-32de7c6681f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.614873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565a79f-1a1d-45ca-94be-32de7c6681f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.618673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 783/1551] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.620804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b734c9-f0fb-4543-a998-7d437e45bd21 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.621091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b734c9-f0fb-4543-a998-7d437e45bd21 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.621365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b734c9-f0fb-4543-a998-7d437e45bd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.626159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565a79f-1a1d-45ca-94be-32de7c6681f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:31.628930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565a79f-1a1d-45ca-94be-32de7c6681f2 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:31.641815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 769/1552] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:26:31.642706 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565a79f-1a1d-45ca-94be-32de7c6681f2 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:31.652576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e3b734c9-f0fb-4543-a998-7d437e45bd21 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.652576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e3b734c9-f0fb-4543-a998-7d437e45bd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.655823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565a79f-1a1d-45ca-94be-32de7c6681f2 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:31.660785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819de118-5d1b-4efe-b413-8ec5c320de96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:31.661357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 784/1553] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.661497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de118-5d1b-4efe-b413-8ec5c320de96 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.661715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de118-5d1b-4efe-b413-8ec5c320de96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.662008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819de118-5d1b-4efe-b413-8ec5c320de96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.662661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a034e4-691e-43ae-ac93-679837b35458 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.662790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819de118-5d1b-4efe-b413-8ec5c320de96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:31.663119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819de118-5d1b-4efe-b413-8ec5c320de96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:31.663248 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a034e4-691e-43ae-ac93-679837b35458 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.663465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a034e4-691e-43ae-ac93-679837b35458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.664283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: WARNING py.warnings [None req-819de118-5d1b-4efe-b413-8ec5c320de96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:26:31.664283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: warnings.warn(msg) Jun 03 15:26:31.664283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: Jun 03 15:26:31.665333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819de118-5d1b-4efe-b413-8ec5c320de96 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:31.681009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 770/1554] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:26:31.682402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fbe46-6915-42e7-9af5-cc38e291d5ef None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.682719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fbe46-6915-42e7-9af5-cc38e291d5ef None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.683268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fbe46-6915-42e7-9af5-cc38e291d5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.684322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-64a034e4-691e-43ae-ac93-679837b35458 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.686191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-64a034e4-691e-43ae-ac93-679837b35458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.695174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 785/1555] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.698653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7ebfb-cc38-44a8-a119-e1a0fb3ab133 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.698938 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7ebfb-cc38-44a8-a119-e1a0fb3ab133 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.699220 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7ebfb-cc38-44a8-a119-e1a0fb3ab133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.704946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-647fbe46-6915-42e7-9af5-cc38e291d5ef None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.706405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-647fbe46-6915-42e7-9af5-cc38e291d5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.716813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 771/1556] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.724764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485dc9b-4a2a-4c6d-996b-4e717ed0eb9e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.725131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485dc9b-4a2a-4c6d-996b-4e717ed0eb9e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.725430 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485dc9b-4a2a-4c6d-996b-4e717ed0eb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.731585 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-71b7ebfb-cc38-44a8-a119-e1a0fb3ab133 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.732916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-71b7ebfb-cc38-44a8-a119-e1a0fb3ab133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.741945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 786/1557] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.743248 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f25ba4-1120-4faf-a26b-b1d8e0313787 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.743531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f25ba4-1120-4faf-a26b-b1d8e0313787 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.743835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f25ba4-1120-4faf-a26b-b1d8e0313787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.745170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e485dc9b-4a2a-4c6d-996b-4e717ed0eb9e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.746395 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e485dc9b-4a2a-4c6d-996b-4e717ed0eb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.755169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 772/1558] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.755906 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485dc9b-4a2a-4c6d-996b-4e717ed0eb9e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:31.764808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-37f25ba4-1120-4faf-a26b-b1d8e0313787 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.766026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-37f25ba4-1120-4faf-a26b-b1d8e0313787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.766401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485dc9b-4a2a-4c6d-996b-4e717ed0eb9e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:31.771593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5810289-dcb9-440c-85dc-2385d8d0dba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:31.772254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5810289-dcb9-440c-85dc-2385d8d0dba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.772533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5810289-dcb9-440c-85dc-2385d8d0dba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.772805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5810289-dcb9-440c-85dc-2385d8d0dba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.773612 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5810289-dcb9-440c-85dc-2385d8d0dba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:31.776757 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5810289-dcb9-440c-85dc-2385d8d0dba8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:31.780506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 787/1559] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.782060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a03ff-fe9b-45ce-8c64-43425932e32e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.782355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a03ff-fe9b-45ce-8c64-43425932e32e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.782689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a03ff-fe9b-45ce-8c64-43425932e32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.789872 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:31.803165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1e2a03ff-fe9b-45ce-8c64-43425932e32e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.804465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1e2a03ff-fe9b-45ce-8c64-43425932e32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.811986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 773/1560] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:31.813301 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c264-3dd7-49b4-969a-cc0f4d016aa6 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.813726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c264-3dd7-49b4-969a-cc0f4d016aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.813942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1c264-3dd7-49b4-969a-cc0f4d016aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.816355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 788/1561] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.818047 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b651cb-7857-4c96-9132-6e099c211a5d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.818461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b651cb-7857-4c96-9132-6e099c211a5d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.818831 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b651cb-7857-4c96-9132-6e099c211a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.836797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-21e1c264-3dd7-49b4-969a-cc0f4d016aa6 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.838989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-21e1c264-3dd7-49b4-969a-cc0f4d016aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.841556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c6b651cb-7857-4c96-9132-6e099c211a5d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.842889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c6b651cb-7857-4c96-9132-6e099c211a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.856296 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:31.920317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 774/1562] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.921570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1c264-3dd7-49b4-969a-cc0f4d016aa6 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:31.928896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 789/1563] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.930423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbb09c-43b0-4e69-b553-d6b7979b3092 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.930762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbb09c-43b0-4e69-b553-d6b7979b3092 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.931428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbb09c-43b0-4e69-b553-d6b7979b3092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.938487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1c264-3dd7-49b4-969a-cc0f4d016aa6 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:31.943239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbede3a-cfbb-4e0c-b490-5be6e8e4d971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:31.943974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbede3a-cfbb-4e0c-b490-5be6e8e4d971 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.944327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbede3a-cfbb-4e0c-b490-5be6e8e4d971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.944670 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbede3a-cfbb-4e0c-b490-5be6e8e4d971 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.945612 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbede3a-cfbb-4e0c-b490-5be6e8e4d971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:31.946131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbede3a-cfbb-4e0c-b490-5be6e8e4d971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:31.947900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbede3a-cfbb-4e0c-b490-5be6e8e4d971 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:31.954759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f9bbb09c-43b0-4e69-b553-d6b7979b3092 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:31.956514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f9bbb09c-43b0-4e69-b553-d6b7979b3092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:31.961288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 775/1564] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:26:31.963027 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbede3a-cfbb-4e0c-b490-5be6e8e4d971 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:31.968189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 790/1565] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:31.972689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbb09c-43b0-4e69-b553-d6b7979b3092 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:31.981039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbede3a-cfbb-4e0c-b490-5be6e8e4d971 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:31.986307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbb09c-43b0-4e69-b553-d6b7979b3092 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:31.987983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b43b2-4ac1-45d2-9775-040402925a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:31.988793 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b43b2-4ac1-45d2-9775-040402925a05 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.989182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b43b2-4ac1-45d2-9775-040402925a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.989557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b43b2-4ac1-45d2-9775-040402925a05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.990401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b43b2-4ac1-45d2-9775-040402925a05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:31.990848 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b43b2-4ac1-45d2-9775-040402925a05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:31.992108 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab526c2-6ed2-4515-8d56-fbeb7a88ca7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:31.992674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401b43b2-4ac1-45d2-9775-040402925a05 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:31.992796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab526c2-6ed2-4515-8d56-fbeb7a88ca7d admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:31.993057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab526c2-6ed2-4515-8d56-fbeb7a88ca7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:31.993334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab526c2-6ed2-4515-8d56-fbeb7a88ca7d admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:31.994065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab526c2-6ed2-4515-8d56-fbeb7a88ca7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:31.996839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab526c2-6ed2-4515-8d56-fbeb7a88ca7d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.005679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 776/1566] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:26:32.006871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b43b2-4ac1-45d2-9775-040402925a05 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.017962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b43b2-4ac1-45d2-9775-040402925a05 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.023954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78712a0e-746d-465a-8345-4b58f009a2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.025572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712a0e-746d-465a-8345-4b58f009a2ce admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.025841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712a0e-746d-465a-8345-4b58f009a2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.026117 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78712a0e-746d-465a-8345-4b58f009a2ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.026985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78712a0e-746d-465a-8345-4b58f009a2ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:32.027362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78712a0e-746d-465a-8345-4b58f009a2ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.029120 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78712a0e-746d-465a-8345-4b58f009a2ce admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.034580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 791/1567] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:31 2022] POST /identity/v3/users => generated 290 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.039398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7008e-bfac-4f11-a97a-cce407504589 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.039909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7008e-bfac-4f11-a97a-cce407504589 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.040272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7008e-bfac-4f11-a97a-cce407504589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.047317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 777/1568] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:26:32.048152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78712a0e-746d-465a-8345-4b58f009a2ce admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.064272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78712a0e-746d-465a-8345-4b58f009a2ce admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.067735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-43f7008e-bfac-4f11-a97a-cce407504589 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:32.069016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-43f7008e-bfac-4f11-a97a-cce407504589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:32.069750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad7e49-1df1-4387-89fa-4c93cffd332e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.070407 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad7e49-1df1-4387-89fa-4c93cffd332e admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.070731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad7e49-1df1-4387-89fa-4c93cffd332e admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.071010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad7e49-1df1-4387-89fa-4c93cffd332e admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.071737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad7e49-1df1-4387-89fa-4c93cffd332e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:32.072575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad7e49-1df1-4387-89fa-4c93cffd332e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.075340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad7e49-1df1-4387-89fa-4c93cffd332e admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.085068 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 792/1569] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:32.085068 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7008e-bfac-4f11-a97a-cce407504589 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.092192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 778/1570] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:26:32.093818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad7e49-1df1-4387-89fa-4c93cffd332e admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.097963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7008e-bfac-4f11-a97a-cce407504589 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.104079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deabce6-02f1-49e2-8229-e317dccaa345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.104869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deabce6-02f1-49e2-8229-e317dccaa345 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.105361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deabce6-02f1-49e2-8229-e317dccaa345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.105705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deabce6-02f1-49e2-8229-e317dccaa345 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.106688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deabce6-02f1-49e2-8229-e317dccaa345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.107853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-5deabce6-02f1-49e2-8229-e317dccaa345 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:26:32.107853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 03 15:26:32.107853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: Jun 03 15:26:32.109337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deabce6-02f1-49e2-8229-e317dccaa345 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.110453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad7e49-1df1-4387-89fa-4c93cffd332e admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.123326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77711-6935-495a-9e83-9a0fb86c5aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.124108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77711-6935-495a-9e83-9a0fb86c5aef admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.124451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77711-6935-495a-9e83-9a0fb86c5aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.124795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77711-6935-495a-9e83-9a0fb86c5aef admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.125598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77711-6935-495a-9e83-9a0fb86c5aef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:32.125983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77711-6935-495a-9e83-9a0fb86c5aef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.127835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77711-6935-495a-9e83-9a0fb86c5aef admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.146638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 793/1571] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.146638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deabce6-02f1-49e2-8229-e317dccaa345 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.148356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 779/1572] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:26:32.149355 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77711-6935-495a-9e83-9a0fb86c5aef admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.161624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77711-6935-495a-9e83-9a0fb86c5aef admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.227531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deabce6-02f1-49e2-8229-e317dccaa345 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.230872 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2cbcf-1d2d-4420-a8b2-679b6b1ddd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.231855 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2cbcf-1d2d-4420-a8b2-679b6b1ddd60 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.232121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2cbcf-1d2d-4420-a8b2-679b6b1ddd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.232473 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2cbcf-1d2d-4420-a8b2-679b6b1ddd60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.233261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2cbcf-1d2d-4420-a8b2-679b6b1ddd60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.233787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ddcbd-0489-4a62-a7ab-df673e102c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.234542 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ddcbd-0489-4a62-a7ab-df673e102c51 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.234899 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ddcbd-0489-4a62-a7ab-df673e102c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.235178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2cbcf-1d2d-4420-a8b2-679b6b1ddd60 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.235305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ddcbd-0489-4a62-a7ab-df673e102c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.236029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ddcbd-0489-4a62-a7ab-df673e102c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.238148 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ddcbd-0489-4a62-a7ab-df673e102c51 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.246883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 780/1573] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:32.248141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2cbcf-1d2d-4420-a8b2-679b6b1ddd60 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.261168 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2cbcf-1d2d-4420-a8b2-679b6b1ddd60 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.266562 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfcdcd-65c2-408b-b6c0-4170bdb945d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.267323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfcdcd-65c2-408b-b6c0-4170bdb945d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.267669 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfcdcd-65c2-408b-b6c0-4170bdb945d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.268014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfcdcd-65c2-408b-b6c0-4170bdb945d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.268781 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfcdcd-65c2-408b-b6c0-4170bdb945d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.269815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 794/1574] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/projects => generated 393 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.270515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ddcbd-0489-4a62-a7ab-df673e102c51 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.271028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfcdcd-65c2-408b-b6c0-4170bdb945d3 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.295387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:32.303879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 781/1575] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.305115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfcdcd-65c2-408b-b6c0-4170bdb945d3 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.329413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:32.348338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ddcbd-0489-4a62-a7ab-df673e102c51 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.380782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfcdcd-65c2-408b-b6c0-4170bdb945d3 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.387478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569661a-0e0d-42b0-a70a-9b0c616871b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.388157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569661a-0e0d-42b0-a70a-9b0c616871b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.388743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569661a-0e0d-42b0-a70a-9b0c616871b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.388961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569661a-0e0d-42b0-a70a-9b0c616871b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.389932 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569661a-0e0d-42b0-a70a-9b0c616871b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.393211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569661a-0e0d-42b0-a70a-9b0c616871b3 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.431752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 782/1576] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.433243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569661a-0e0d-42b0-a70a-9b0c616871b3 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.433982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca51bf-d3a6-4530-946a-794cf74ce525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.434865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca51bf-d3a6-4530-946a-794cf74ce525 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.435236 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca51bf-d3a6-4530-946a-794cf74ce525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.435647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca51bf-d3a6-4530-946a-794cf74ce525 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.436403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca51bf-d3a6-4530-946a-794cf74ce525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:32.436714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca51bf-d3a6-4530-946a-794cf74ce525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.438943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca51bf-d3a6-4530-946a-794cf74ce525 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.463760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 795/1577] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:26:32.463760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca51bf-d3a6-4530-946a-794cf74ce525 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.472963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca51bf-d3a6-4530-946a-794cf74ce525 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.524405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569661a-0e0d-42b0-a70a-9b0c616871b3 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.530919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf223-f47e-400a-a6fb-751f08baf0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.531748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf223-f47e-400a-a6fb-751f08baf0b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.532132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf223-f47e-400a-a6fb-751f08baf0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.532581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcf223-f47e-400a-a6fb-751f08baf0b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.533388 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcf223-f47e-400a-a6fb-751f08baf0b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.534683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: WARNING py.warnings [None req-64fcf223-f47e-400a-a6fb-751f08baf0b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:26:32.534683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: warnings.warn(msg) Jun 03 15:26:32.534683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: Jun 03 15:26:32.536414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcf223-f47e-400a-a6fb-751f08baf0b3 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.564244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ee744-1712-433e-ae66-e5b2db4430f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.564977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ee744-1712-433e-ae66-e5b2db4430f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.565227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ee744-1712-433e-ae66-e5b2db4430f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.565506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ee744-1712-433e-ae66-e5b2db4430f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.566202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ee744-1712-433e-ae66-e5b2db4430f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.568198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ee744-1712-433e-ae66-e5b2db4430f6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.584288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 783/1578] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/users => generated 283 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.585283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf223-f47e-400a-a6fb-751f08baf0b3 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.609732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 796/1579] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/projects => generated 373 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.609935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcf223-f47e-400a-a6fb-751f08baf0b3 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.610031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ee744-1712-433e-ae66-e5b2db4430f6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.682486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167cd2b-6de0-4414-8534-e182101a79a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.683364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167cd2b-6de0-4414-8534-e182101a79a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.683746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167cd2b-6de0-4414-8534-e182101a79a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.684121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9167cd2b-6de0-4414-8534-e182101a79a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.685174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167cd2b-6de0-4414-8534-e182101a79a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.686945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9167cd2b-6de0-4414-8534-e182101a79a4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.687493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ee744-1712-433e-ae66-e5b2db4430f6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.693122 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae1518-2ebd-4826-ab52-4289893d453b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.693803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae1518-2ebd-4826-ab52-4289893d453b admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.694088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae1518-2ebd-4826-ab52-4289893d453b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.694366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae1518-2ebd-4826-ab52-4289893d453b admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.695254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae1518-2ebd-4826-ab52-4289893d453b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.698160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae1518-2ebd-4826-ab52-4289893d453b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.700314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 784/1580] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:32.701514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167cd2b-6de0-4414-8534-e182101a79a4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.713319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9167cd2b-6de0-4414-8534-e182101a79a4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.722256 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25321fc2-9d02-4911-91ca-28117461dcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.723153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25321fc2-9d02-4911-91ca-28117461dcd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.723617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25321fc2-9d02-4911-91ca-28117461dcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.724158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25321fc2-9d02-4911-91ca-28117461dcd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.725032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25321fc2-9d02-4911-91ca-28117461dcd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.727330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25321fc2-9d02-4911-91ca-28117461dcd5 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.732827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 797/1581] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.733989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae1518-2ebd-4826-ab52-4289893d453b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.738033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:32.741551 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:32.750892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae1518-2ebd-4826-ab52-4289893d453b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.756745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927cb0a-6a3e-4350-93d2-e0ddebd5f80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.756997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 785/1582] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/users => generated 287 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.757928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927cb0a-6a3e-4350-93d2-e0ddebd5f80f admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.758169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25321fc2-9d02-4911-91ca-28117461dcd5 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.758517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927cb0a-6a3e-4350-93d2-e0ddebd5f80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.758961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927cb0a-6a3e-4350-93d2-e0ddebd5f80f admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.761814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927cb0a-6a3e-4350-93d2-e0ddebd5f80f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.764253 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927cb0a-6a3e-4350-93d2-e0ddebd5f80f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.776056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25321fc2-9d02-4911-91ca-28117461dcd5 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.782570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a219f-c89d-41ab-b3c1-40ea89bd3ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.783402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a219f-c89d-41ab-b3c1-40ea89bd3ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.783606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a219f-c89d-41ab-b3c1-40ea89bd3ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.783880 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a219f-c89d-41ab-b3c1-40ea89bd3ca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.784652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a219f-c89d-41ab-b3c1-40ea89bd3ca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.786726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a219f-c89d-41ab-b3c1-40ea89bd3ca0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.795436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 798/1583] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/users => generated 295 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.796903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927cb0a-6a3e-4350-93d2-e0ddebd5f80f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.810505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927cb0a-6a3e-4350-93d2-e0ddebd5f80f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.816337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261803a-395a-43bd-aaa4-dea05f3ee394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.817167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261803a-395a-43bd-aaa4-dea05f3ee394 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.817587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261803a-395a-43bd-aaa4-dea05f3ee394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.818157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261803a-395a-43bd-aaa4-dea05f3ee394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.819273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261803a-395a-43bd-aaa4-dea05f3ee394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.821705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6261803a-395a-43bd-aaa4-dea05f3ee394 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.835849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 799/1584] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:32.837106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261803a-395a-43bd-aaa4-dea05f3ee394 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.849697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 786/1585] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.851974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261803a-395a-43bd-aaa4-dea05f3ee394 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.855147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a219f-c89d-41ab-b3c1-40ea89bd3ca0 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.947418 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a219f-c89d-41ab-b3c1-40ea89bd3ca0 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:32.951533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b830d-e6a5-4eb8-b3e8-a9d0c0985aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.952785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b830d-e6a5-4eb8-b3e8-a9d0c0985aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.952980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1aa35c-b9df-4bdf-8f1a-1d5031c5f1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:32.953111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b830d-e6a5-4eb8-b3e8-a9d0c0985aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.953347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b830d-e6a5-4eb8-b3e8-a9d0c0985aa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.954102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1aa35c-b9df-4bdf-8f1a-1d5031c5f1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:32.954236 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b830d-e6a5-4eb8-b3e8-a9d0c0985aa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.956799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1aa35c-b9df-4bdf-8f1a-1d5031c5f1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:32.957169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1aa35c-b9df-4bdf-8f1a-1d5031c5f1ae admin admin] PATH_INFO: `/v3/projects/683acaeef18d43b4ac3daeb60e3a8aea/users/bdd62219456547db82c2f5416d4f44f6/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:32.957708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b830d-e6a5-4eb8-b3e8-a9d0c0985aa4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:32.988377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1aa35c-b9df-4bdf-8f1a-1d5031c5f1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=683acaeef18d43b4ac3daeb60e3a8aea, user_id=bdd62219456547db82c2f5416d4f44f6, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:32.989597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 800/1586] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:32 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:32.990572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b830d-e6a5-4eb8-b3e8-a9d0c0985aa4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:32.992864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1aa35c-b9df-4bdf-8f1a-1d5031c5f1ae admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.010639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b830d-e6a5-4eb8-b3e8-a9d0c0985aa4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.022567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 787/1587] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:32 2022] PUT /identity/v3/projects/683acaeef18d43b4ac3daeb60e3a8aea/users/bdd62219456547db82c2f5416d4f44f6/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:33.023494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1aa35c-b9df-4bdf-8f1a-1d5031c5f1ae admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.120034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4e773-2c35-43e6-b605-72992aca2702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.121046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4e773-2c35-43e6-b605-72992aca2702 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.121493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4e773-2c35-43e6-b605-72992aca2702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.121939 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4e773-2c35-43e6-b605-72992aca2702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.122737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4e773-2c35-43e6-b605-72992aca2702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.124368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4e773-2c35-43e6-b605-72992aca2702 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.129345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1aa35c-b9df-4bdf-8f1a-1d5031c5f1ae admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.140137 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837758c-c760-4c6b-8fce-a38e0ceb4cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.140810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837758c-c760-4c6b-8fce-a38e0ceb4cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.141094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837758c-c760-4c6b-8fce-a38e0ceb4cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.142390 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837758c-c760-4c6b-8fce-a38e0ceb4cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.143164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837758c-c760-4c6b-8fce-a38e0ceb4cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.145042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837758c-c760-4c6b-8fce-a38e0ceb4cf9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.152041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 801/1588] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:33.153000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4e773-2c35-43e6-b605-72992aca2702 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.169864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 788/1589] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:33.171378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837758c-c760-4c6b-8fce-a38e0ceb4cf9 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.176178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4e773-2c35-43e6-b605-72992aca2702 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.183702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837758c-c760-4c6b-8fce-a38e0ceb4cf9 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.190053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545193-d02d-4d1d-9f55-cf63ab2361f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.191042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545193-d02d-4d1d-9f55-cf63ab2361f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.191227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170a9c9-edd6-4e5c-957a-821dbd80a4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.191665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545193-d02d-4d1d-9f55-cf63ab2361f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.192044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545193-d02d-4d1d-9f55-cf63ab2361f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.192396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a9c9-edd6-4e5c-957a-821dbd80a4ba admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.192844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a9c9-edd6-4e5c-957a-821dbd80a4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.193084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb545193-d02d-4d1d-9f55-cf63ab2361f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.194476 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170a9c9-edd6-4e5c-957a-821dbd80a4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.195015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb545193-d02d-4d1d-9f55-cf63ab2361f5 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.195602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170a9c9-edd6-4e5c-957a-821dbd80a4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.197689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170a9c9-edd6-4e5c-957a-821dbd80a4ba admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.213046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 802/1590] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:33.214334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170a9c9-edd6-4e5c-957a-821dbd80a4ba admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.224231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 789/1591] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:33.224231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545193-d02d-4d1d-9f55-cf63ab2361f5 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.229842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170a9c9-edd6-4e5c-957a-821dbd80a4ba admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.236864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9f0c3-2aff-4ca8-b9e3-dc6b7ca5d660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.238009 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9f0c3-2aff-4ca8-b9e3-dc6b7ca5d660 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.238513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9f0c3-2aff-4ca8-b9e3-dc6b7ca5d660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.239181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9f0c3-2aff-4ca8-b9e3-dc6b7ca5d660 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.240155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c9f0c3-2aff-4ca8-b9e3-dc6b7ca5d660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.242584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c9f0c3-2aff-4ca8-b9e3-dc6b7ca5d660 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.254831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545193-d02d-4d1d-9f55-cf63ab2361f5 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.262015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27dd446-4bde-4384-a3e9-d1b92b0dff66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.263514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dd446-4bde-4384-a3e9-d1b92b0dff66 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.263514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dd446-4bde-4384-a3e9-d1b92b0dff66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.263721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27dd446-4bde-4384-a3e9-d1b92b0dff66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.264508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dd446-4bde-4384-a3e9-d1b92b0dff66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.266335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27dd446-4bde-4384-a3e9-d1b92b0dff66 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.280794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 790/1592] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:33.282270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06e38d-ad1c-4870-bc0f-920ba71aa33d None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.282741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06e38d-ad1c-4870-bc0f-920ba71aa33d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.283102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06e38d-ad1c-4870-bc0f-920ba71aa33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.287206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 803/1593] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:33 2022] POST /identity/v3/users => generated 293 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:33.295965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9f0c3-2aff-4ca8-b9e3-dc6b7ca5d660 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.307152 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9f0c3-2aff-4ca8-b9e3-dc6b7ca5d660 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.308917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-4a06e38d-ad1c-4870-bc0f-920ba71aa33d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:33.310711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-4a06e38d-ad1c-4870-bc0f-920ba71aa33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:33.312591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3157d-8de4-4789-91eb-182c5e102e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.313507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3157d-8de4-4789-91eb-182c5e102e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.313869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3157d-8de4-4789-91eb-182c5e102e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.314212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3157d-8de4-4789-91eb-182c5e102e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.315055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3157d-8de4-4789-91eb-182c5e102e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.316968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3157d-8de4-4789-91eb-182c5e102e84 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.322691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 791/1594] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:33.322691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06e38d-ad1c-4870-bc0f-920ba71aa33d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.332626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06e38d-ad1c-4870-bc0f-920ba71aa33d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.335226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 804/1595] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:33.336165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3157d-8de4-4789-91eb-182c5e102e84 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.339978 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af566efc-1195-452a-b7b5-672b4d1bfd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.340561 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af566efc-1195-452a-b7b5-672b4d1bfd9c admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.340850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af566efc-1195-452a-b7b5-672b4d1bfd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.341120 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af566efc-1195-452a-b7b5-672b4d1bfd9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.341860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af566efc-1195-452a-b7b5-672b4d1bfd9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.343521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af566efc-1195-452a-b7b5-672b4d1bfd9c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.348402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3157d-8de4-4789-91eb-182c5e102e84 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.353985 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177b31-94e9-4ece-8780-b7e4e31ed464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.354701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177b31-94e9-4ece-8780-b7e4e31ed464 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.354981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177b31-94e9-4ece-8780-b7e4e31ed464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.355254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177b31-94e9-4ece-8780-b7e4e31ed464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.355981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01177b31-94e9-4ece-8780-b7e4e31ed464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.357859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01177b31-94e9-4ece-8780-b7e4e31ed464 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.360260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 792/1596] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:33.361409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af566efc-1195-452a-b7b5-672b4d1bfd9c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.371945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af566efc-1195-452a-b7b5-672b4d1bfd9c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.375356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 805/1597] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:33.375356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177b31-94e9-4ece-8780-b7e4e31ed464 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.378075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81707aea-4436-4b3c-896e-8330328ee782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.379017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707aea-4436-4b3c-896e-8330328ee782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.379294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707aea-4436-4b3c-896e-8330328ee782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.380048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707aea-4436-4b3c-896e-8330328ee782 admin admin] PATH_INFO: `/v3/projects/03e032dba06e4185b897268d78a35290/users/4de5b16e95b542428d827bba01fcc377/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.383648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177b31-94e9-4ece-8780-b7e4e31ed464 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.388682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b7c87-2177-4343-86eb-b0041cbb503b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.389509 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7c87-2177-4343-86eb-b0041cbb503b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.389789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7c87-2177-4343-86eb-b0041cbb503b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.390061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74b7c87-2177-4343-86eb-b0041cbb503b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.390830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b7c87-2177-4343-86eb-b0041cbb503b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.390949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:33.392639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74b7c87-2177-4343-86eb-b0041cbb503b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.398598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:33.405671 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81707aea-4436-4b3c-896e-8330328ee782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e032dba06e4185b897268d78a35290, user_id=4de5b16e95b542428d827bba01fcc377, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.410080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 806/1598] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:33.410080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b7c87-2177-4343-86eb-b0041cbb503b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.414580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81707aea-4436-4b3c-896e-8330328ee782 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.422645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74b7c87-2177-4343-86eb-b0041cbb503b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.428530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e25e3-df69-4647-8bf5-dfe69bf23723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.429667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e25e3-df69-4647-8bf5-dfe69bf23723 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.429953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e25e3-df69-4647-8bf5-dfe69bf23723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.430457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e25e3-df69-4647-8bf5-dfe69bf23723 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.431472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e25e3-df69-4647-8bf5-dfe69bf23723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.433381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e25e3-df69-4647-8bf5-dfe69bf23723 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.436413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 793/1599] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:33 2022] PUT /identity/v3/projects/03e032dba06e4185b897268d78a35290/users/4de5b16e95b542428d827bba01fcc377/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:33.447983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81707aea-4436-4b3c-896e-8330328ee782 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.449582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 807/1600] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:33.449691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e25e3-df69-4647-8bf5-dfe69bf23723 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.461668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e25e3-df69-4647-8bf5-dfe69bf23723 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.526810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81707aea-4436-4b3c-896e-8330328ee782 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.531208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa712b-84c5-488d-ba35-01b562f3cc9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.531867 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa712b-84c5-488d-ba35-01b562f3cc9a service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.532151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa712b-84c5-488d-ba35-01b562f3cc9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.532434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa712b-84c5-488d-ba35-01b562f3cc9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.550370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af1cbe-7f75-4ac8-b4ea-32280e110015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.557824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af1cbe-7f75-4ac8-b4ea-32280e110015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.557824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af1cbe-7f75-4ac8-b4ea-32280e110015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.561729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af1cbe-7f75-4ac8-b4ea-32280e110015 admin admin] PATH_INFO: `/v3/projects/a402ec1153624838be07cda58279f2a4/users/127a6938bc5c4416a9a502d9064344cd/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.588988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af1cbe-7f75-4ac8-b4ea-32280e110015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a402ec1153624838be07cda58279f2a4, user_id=127a6938bc5c4416a9a502d9064344cd, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.590235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-14af1cbe-7f75-4ac8-b4ea-32280e110015 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:26:33.590235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 03 15:26:33.590235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: Jun 03 15:26:33.593249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14af1cbe-7f75-4ac8-b4ea-32280e110015 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.621106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 808/1601] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:33 2022] PUT /identity/v3/projects/a402ec1153624838be07cda58279f2a4/users/127a6938bc5c4416a9a502d9064344cd/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:33.622338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af1cbe-7f75-4ac8-b4ea-32280e110015 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.658013 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa712b-84c5-488d-ba35-01b562f3cc9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.659622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfa712b-84c5-488d-ba35-01b562f3cc9a service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.677871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 794/1602] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:33.678856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa712b-84c5-488d-ba35-01b562f3cc9a service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.692221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa712b-84c5-488d-ba35-01b562f3cc9a service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.716937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af1cbe-7f75-4ac8-b4ea-32280e110015 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.724970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88948281-5d9d-462d-97f0-3136b5c13b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.726194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88948281-5d9d-462d-97f0-3136b5c13b5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.726840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88948281-5d9d-462d-97f0-3136b5c13b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.727306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88948281-5d9d-462d-97f0-3136b5c13b5c admin admin] PATH_INFO: `/v3/projects/b710b999eb884a96a85e9c6debe49306/users/2e3ea44b40d9419aa51e0fa691d8dce9/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.754030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88948281-5d9d-462d-97f0-3136b5c13b5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b710b999eb884a96a85e9c6debe49306, user_id=2e3ea44b40d9419aa51e0fa691d8dce9, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.763952 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88948281-5d9d-462d-97f0-3136b5c13b5c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.786973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 809/1603] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:33 2022] PUT /identity/v3/projects/b710b999eb884a96a85e9c6debe49306/users/2e3ea44b40d9419aa51e0fa691d8dce9/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:33.788103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88948281-5d9d-462d-97f0-3136b5c13b5c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.792567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3ea90-88e6-433c-a8ef-7c280e7273d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.794039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3ea90-88e6-433c-a8ef-7c280e7273d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.794414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3ea90-88e6-433c-a8ef-7c280e7273d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.794837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce3ea90-88e6-433c-a8ef-7c280e7273d9 admin admin] PATH_INFO: `/v3/projects/1b930cadb2dc4e49947db0514528617d/users/47d7587d10b3487688ed251a85b83c97/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.817262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3ea90-88e6-433c-a8ef-7c280e7273d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b930cadb2dc4e49947db0514528617d, user_id=47d7587d10b3487688ed251a85b83c97, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.820719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce3ea90-88e6-433c-a8ef-7c280e7273d9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.843029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 795/1604] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:33 2022] PUT /identity/v3/projects/1b930cadb2dc4e49947db0514528617d/users/47d7587d10b3487688ed251a85b83c97/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:33.843916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3ea90-88e6-433c-a8ef-7c280e7273d9 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:33.872149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:33.875753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:33.889964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88948281-5d9d-462d-97f0-3136b5c13b5c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.963677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce3ea90-88e6-433c-a8ef-7c280e7273d9 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:33.970094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc8fde-44d8-42d5-a6cf-eb1b8df5b780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:33.971738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc8fde-44d8-42d5-a6cf-eb1b8df5b780 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.972256 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc8fde-44d8-42d5-a6cf-eb1b8df5b780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.972697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc8fde-44d8-42d5-a6cf-eb1b8df5b780 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:33.973762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc8fde-44d8-42d5-a6cf-eb1b8df5b780 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:33.975841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dc8fde-44d8-42d5-a6cf-eb1b8df5b780 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:33.993214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 796/1605] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:33.995531 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ee972-09fd-4de1-9fd4-be9fc2d92ded None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:33.995876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ee972-09fd-4de1-9fd4-be9fc2d92ded None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:33.996250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ee972-09fd-4de1-9fd4-be9fc2d92ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.013895 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6c567-422b-482d-a962-2b50aa141f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:34.015570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6c567-422b-482d-a962-2b50aa141f88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.015933 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6c567-422b-482d-a962-2b50aa141f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.016213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6c567-422b-482d-a962-2b50aa141f88 admin admin] PATH_INFO: `/v3/projects/8e608c77c1ac4f4d9a62d95d45ad9d5a/users/9d90b1fcab1c4477b4b2d6c6c219c91b/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.035332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e76ee972-09fd-4de1-9fd4-be9fc2d92ded None None] MFA Rules not processed for user `4de5b16e95b542428d827bba01fcc377`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:34.036377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e76ee972-09fd-4de1-9fd4-be9fc2d92ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:34.038452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6c567-422b-482d-a962-2b50aa141f88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e608c77c1ac4f4d9a62d95d45ad9d5a, user_id=9d90b1fcab1c4477b4b2d6c6c219c91b, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:34.041837 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6c567-422b-482d-a962-2b50aa141f88 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:34.064147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 810/1606] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:33 2022] PUT /identity/v3/projects/8e608c77c1ac4f4d9a62d95d45ad9d5a/users/9d90b1fcab1c4477b4b2d6c6c219c91b/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:34.065663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045290-a142-430b-8fb2-b56bbec5cdb7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.066129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045290-a142-430b-8fb2-b56bbec5cdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.066415 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f045290-a142-430b-8fb2-b56bbec5cdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.101537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1f045290-a142-430b-8fb2-b56bbec5cdb7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:34.102920 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1f045290-a142-430b-8fb2-b56bbec5cdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:34.170116 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 797/1607] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:34.171643 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3da98-ca77-442d-82f5-66a14ba45f46 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.173170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3da98-ca77-442d-82f5-66a14ba45f46 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.173451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3da98-ca77-442d-82f5-66a14ba45f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.186873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 811/1608] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:34.188393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae9643-b2b9-4d06-ae23-e4287c658af4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.188791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae9643-b2b9-4d06-ae23-e4287c658af4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.189171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae9643-b2b9-4d06-ae23-e4287c658af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.221550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-beae9643-b2b9-4d06-ae23-e4287c658af4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:34.224113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-beae9643-b2b9-4d06-ae23-e4287c658af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:34.228156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-eda3da98-ca77-442d-82f5-66a14ba45f46 None None] MFA Rules not processed for user `2e3ea44b40d9419aa51e0fa691d8dce9`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:34.229574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-eda3da98-ca77-442d-82f5-66a14ba45f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:34.237463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 812/1609] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:34.239160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae9643-b2b9-4d06-ae23-e4287c658af4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:34.265136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae9643-b2b9-4d06-ae23-e4287c658af4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:34.271455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4336d-7b22-4d33-82d7-549962b5abcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:34.273520 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4336d-7b22-4d33-82d7-549962b5abcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.273957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4336d-7b22-4d33-82d7-549962b5abcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.274686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4336d-7b22-4d33-82d7-549962b5abcb admin admin] PATH_INFO: `/v3/projects/3e8f48e1939245eaa67bcd6a27bc7cda/users/193a35a91a47410f8b12cb2e84b33e1f/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.300767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4336d-7b22-4d33-82d7-549962b5abcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e8f48e1939245eaa67bcd6a27bc7cda, user_id=193a35a91a47410f8b12cb2e84b33e1f, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:34.304550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d4336d-7b22-4d33-82d7-549962b5abcb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:34.335134 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 813/1610] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:34 2022] PUT /identity/v3/projects/3e8f48e1939245eaa67bcd6a27bc7cda/users/193a35a91a47410f8b12cb2e84b33e1f/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:34.341191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 798/1611] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:34.341191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3da98-ca77-442d-82f5-66a14ba45f46 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:34.341445 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb604c0-b4e6-423e-91dd-789c2b0b3de7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.341445 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb604c0-b4e6-423e-91dd-789c2b0b3de7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.342239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb604c0-b4e6-423e-91dd-789c2b0b3de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.365918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3da98-ca77-442d-82f5-66a14ba45f46 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:34.410907 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5bb604c0-b4e6-423e-91dd-789c2b0b3de7 None None] MFA Rules not processed for user `9d90b1fcab1c4477b4b2d6c6c219c91b`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:34.412149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5bb604c0-b4e6-423e-91dd-789c2b0b3de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:34.419210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:34.454749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a95af-0f2f-43fa-801d-20dbd4a20b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:34.455808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a95af-0f2f-43fa-801d-20dbd4a20b02 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.456314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a95af-0f2f-43fa-801d-20dbd4a20b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.456790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a95af-0f2f-43fa-801d-20dbd4a20b02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.457986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a95af-0f2f-43fa-801d-20dbd4a20b02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:34.460386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a95af-0f2f-43fa-801d-20dbd4a20b02 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:34.496875 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:34.513442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 799/1612] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:34 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:34.514228 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a95af-0f2f-43fa-801d-20dbd4a20b02 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:34.517100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-444a95af-0f2f-43fa-801d-20dbd4a20b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:34.578153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 814/1613] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:34.579171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb604c0-b4e6-423e-91dd-789c2b0b3de7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:34.580585 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5bb604c0-b4e6-423e-91dd-789c2b0b3de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:34.637060 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a95af-0f2f-43fa-801d-20dbd4a20b02 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:34.643239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470904b-7bd0-4613-b0f7-2efa0c127d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:34.644091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470904b-7bd0-4613-b0f7-2efa0c127d29 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.644472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470904b-7bd0-4613-b0f7-2efa0c127d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.644850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470904b-7bd0-4613-b0f7-2efa0c127d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.715259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb604c0-b4e6-423e-91dd-789c2b0b3de7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:34.725997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399b2df1-605b-44b0-b744-0e39d80a6169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:34.726752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b2df1-605b-44b0-b744-0e39d80a6169 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.727110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b2df1-605b-44b0-b744-0e39d80a6169 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.727408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b2df1-605b-44b0-b744-0e39d80a6169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.796883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470904b-7bd0-4613-b0f7-2efa0c127d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:34.798371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470904b-7bd0-4613-b0f7-2efa0c127d29 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:34.819364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b2df1-605b-44b0-b744-0e39d80a6169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:34.820845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399b2df1-605b-44b0-b744-0e39d80a6169 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:34.827527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 800/1614] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:34.828894 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e427ac-5e14-492d-8d15-61086e1a398e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.829186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e427ac-5e14-492d-8d15-61086e1a398e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.829469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e427ac-5e14-492d-8d15-61086e1a398e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.846748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 815/1615] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:34.848149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797716b2-0ef4-42e6-82c6-684fd12b5afe None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.848442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797716b2-0ef4-42e6-82c6-684fd12b5afe None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.848720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797716b2-0ef4-42e6-82c6-684fd12b5afe None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.850271 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 816/1616] 10.222.0.93 () {54 vars in 961 bytes} [Fri Jun 3 15:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:26:34.852373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797716b2-0ef4-42e6-82c6-684fd12b5afe None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:34.859657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-a3e427ac-5e14-492d-8d15-61086e1a398e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:34.860898 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a3e427ac-5e14-492d-8d15-61086e1a398e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:34.875373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797716b2-0ef4-42e6-82c6-684fd12b5afe None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:34.881346 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 801/1617] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:34.882194 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e427ac-5e14-492d-8d15-61086e1a398e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:34.896684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf35cba-0f59-4aa2-88ae-a19b176ace1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:34.897373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf35cba-0f59-4aa2-88ae-a19b176ace1c admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.897657 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf35cba-0f59-4aa2-88ae-a19b176ace1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.897934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf35cba-0f59-4aa2-88ae-a19b176ace1c admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.898677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf35cba-0f59-4aa2-88ae-a19b176ace1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:34.901502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf35cba-0f59-4aa2-88ae-a19b176ace1c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:34.915270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e427ac-5e14-492d-8d15-61086e1a398e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:34.924693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc261710-0aa4-4b8e-ba5e-6b61d87847c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:34.925505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc261710-0aa4-4b8e-ba5e-6b61d87847c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.925796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc261710-0aa4-4b8e-ba5e-6b61d87847c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.926074 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc261710-0aa4-4b8e-ba5e-6b61d87847c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.926874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc261710-0aa4-4b8e-ba5e-6b61d87847c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:34.928935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc261710-0aa4-4b8e-ba5e-6b61d87847c9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:34.942648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 817/1618] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:34 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:34.944003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e5d8b-0bf5-4f14-a81b-788d629c4e30 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.944308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e5d8b-0bf5-4f14-a81b-788d629c4e30 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.944844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88e5d8b-0bf5-4f14-a81b-788d629c4e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:34.994208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 802/1619] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:34 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:34.996510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f98514-de27-44fa-8574-2953e27a3999 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:34.996858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f98514-de27-44fa-8574-2953e27a3999 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:34.997907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f98514-de27-44fa-8574-2953e27a3999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.005240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c88e5d8b-0bf5-4f14-a81b-788d629c4e30 None None] MFA Rules not processed for user `bdd62219456547db82c2f5416d4f44f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:35.007667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c88e5d8b-0bf5-4f14-a81b-788d629c4e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:35.034560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-11f98514-de27-44fa-8574-2953e27a3999 None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:35.036733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-11f98514-de27-44fa-8574-2953e27a3999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:35.120908 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:35.166288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 818/1620] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:35.167386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e5d8b-0bf5-4f14-a81b-788d629c4e30 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:35.179160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:35.198475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 803/1621] 10.222.0.93 () {58 vars in 1058 bytes} [Fri Jun 3 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:35.200272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f98514-de27-44fa-8574-2953e27a3999 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:35.287180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88e5d8b-0bf5-4f14-a81b-788d629c4e30 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:35.293100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe9921-69b9-4113-9ed1-3193ee43ae27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:35.294478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe9921-69b9-4113-9ed1-3193ee43ae27 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:35.295484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe9921-69b9-4113-9ed1-3193ee43ae27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:35.295484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebe9921-69b9-4113-9ed1-3193ee43ae27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.321993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f98514-de27-44fa-8574-2953e27a3999 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:35.324085 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe9921-69b9-4113-9ed1-3193ee43ae27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:35.326778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebe9921-69b9-4113-9ed1-3193ee43ae27 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:35.335420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582f802-8221-4364-9a37-2505b52ce8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:35.340108 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 819/1622] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:35.346009 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe9921-69b9-4113-9ed1-3193ee43ae27 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:35.355179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582f802-8221-4364-9a37-2505b52ce8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:35.355475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582f802-8221-4364-9a37-2505b52ce8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:35.355767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582f802-8221-4364-9a37-2505b52ce8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.357696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebe9921-69b9-4113-9ed1-3193ee43ae27 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:35.360016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582f802-8221-4364-9a37-2505b52ce8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:35.364385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae8fe7-99f9-4d77-8523-66ed62c991fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:35.365082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae8fe7-99f9-4d77-8523-66ed62c991fa admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:35.365345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582f802-8221-4364-9a37-2505b52ce8d3 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:35.365504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae8fe7-99f9-4d77-8523-66ed62c991fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:35.365781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae8fe7-99f9-4d77-8523-66ed62c991fa admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.369686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae8fe7-99f9-4d77-8523-66ed62c991fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:35.372280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae8fe7-99f9-4d77-8523-66ed62c991fa admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:35.396814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 804/1623] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:35.398936 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec56e46-48cc-44b6-8aef-3d8a80407265 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:35.399650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec56e46-48cc-44b6-8aef-3d8a80407265 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:35.400483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec56e46-48cc-44b6-8aef-3d8a80407265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.416741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 820/1624] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:35 2022] POST /identity/v3/users => generated 292 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:35.424662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae8fe7-99f9-4d77-8523-66ed62c991fa admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:35.440968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae8fe7-99f9-4d77-8523-66ed62c991fa admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:35.448560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9d6f2-7240-4be4-968c-633db7b02f83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:35.449229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9d6f2-7240-4be4-968c-633db7b02f83 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:35.449511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9d6f2-7240-4be4-968c-633db7b02f83 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:35.449830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9d6f2-7240-4be4-968c-633db7b02f83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.449923 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0ec56e46-48cc-44b6-8aef-3d8a80407265 None None] MFA Rules not processed for user `bdd62219456547db82c2f5416d4f44f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:35.453997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0ec56e46-48cc-44b6-8aef-3d8a80407265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:35.476126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 805/1625] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:35.484416 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec56e46-48cc-44b6-8aef-3d8a80407265 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:35.528657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec56e46-48cc-44b6-8aef-3d8a80407265 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:35.546999 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53faa2ec-9548-42d6-b374-78ad0f951006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:35.547915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faa2ec-9548-42d6-b374-78ad0f951006 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:35.548206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faa2ec-9548-42d6-b374-78ad0f951006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:35.557802 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53faa2ec-9548-42d6-b374-78ad0f951006 admin admin] PATH_INFO: `/v3/projects/f6238785c8c04ae1841ee859cb3ebbc8/users/28f76dcdfc69420cb20a0ba2f6462449/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.563722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9d6f2-7240-4be4-968c-633db7b02f83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:35.569555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9d6f2-7240-4be4-968c-633db7b02f83 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:35.594026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53faa2ec-9548-42d6-b374-78ad0f951006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6238785c8c04ae1841ee859cb3ebbc8, user_id=28f76dcdfc69420cb20a0ba2f6462449, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:35.601451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53faa2ec-9548-42d6-b374-78ad0f951006 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:35.604168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 821/1626] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:35.605490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9d6f2-7240-4be4-968c-633db7b02f83 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:35.637200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 806/1627] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:35 2022] PUT /identity/v3/projects/f6238785c8c04ae1841ee859cb3ebbc8/users/28f76dcdfc69420cb20a0ba2f6462449/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:35.644770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bae565-2769-4661-a690-c5bab5c9eb63 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:35.644770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bae565-2769-4661-a690-c5bab5c9eb63 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:35.644770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bae565-2769-4661-a690-c5bab5c9eb63 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.645193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9d6f2-7240-4be4-968c-633db7b02f83 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:35.646360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 807/1628] 10.222.0.93 () {54 vars in 961 bytes} [Fri Jun 3 15:26:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:26:35.647309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bae565-2769-4661-a690-c5bab5c9eb63 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:35.851840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bae565-2769-4661-a690-c5bab5c9eb63 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:35.857499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12809f3-504c-4da4-98b8-a595363324e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:35.863012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12809f3-504c-4da4-98b8-a595363324e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:35.863414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12809f3-504c-4da4-98b8-a595363324e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:35.863567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4145a6-18ca-4a1f-ad5a-f032100e0ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:35.864014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12809f3-504c-4da4-98b8-a595363324e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.864868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12809f3-504c-4da4-98b8-a595363324e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:35.866329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4145a6-18ca-4a1f-ad5a-f032100e0ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:35.866676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4145a6-18ca-4a1f-ad5a-f032100e0ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:35.867009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4145a6-18ca-4a1f-ad5a-f032100e0ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.867446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12809f3-504c-4da4-98b8-a595363324e7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:35.867780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4145a6-18ca-4a1f-ad5a-f032100e0ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:35.872402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4145a6-18ca-4a1f-ad5a-f032100e0ed1 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:35.899716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 808/1629] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:35.901218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05adc-04f6-4f51-a6a5-86ab3cccb864 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:35.903745 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05adc-04f6-4f51-a6a5-86ab3cccb864 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:35.904138 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb05adc-04f6-4f51-a6a5-86ab3cccb864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.905217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 822/1630] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:35.909443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12809f3-504c-4da4-98b8-a595363324e7 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:35.928597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12809f3-504c-4da4-98b8-a595363324e7 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:35.942302 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01806cb-2128-4950-b35f-c6efab2a7977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:35.944793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01806cb-2128-4950-b35f-c6efab2a7977 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:35.946385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01806cb-2128-4950-b35f-c6efab2a7977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:35.949779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01806cb-2128-4950-b35f-c6efab2a7977 admin admin] PATH_INFO: `/v3/projects/f6238785c8c04ae1841ee859cb3ebbc8/users/28f76dcdfc69420cb20a0ba2f6462449/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:35.953355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01806cb-2128-4950-b35f-c6efab2a7977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6238785c8c04ae1841ee859cb3ebbc8, user_id=28f76dcdfc69420cb20a0ba2f6462449, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:35.960033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-adb05adc-04f6-4f51-a6a5-86ab3cccb864 None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:35.960690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01806cb-2128-4950-b35f-c6efab2a7977 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:35.964426 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-adb05adc-04f6-4f51-a6a5-86ab3cccb864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:35.999772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 823/1631] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:35 2022] PUT /identity/v3/projects/f6238785c8c04ae1841ee859cb3ebbc8/users/28f76dcdfc69420cb20a0ba2f6462449/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:35.999772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01806cb-2128-4950-b35f-c6efab2a7977 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:36.027316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:36.041065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:36.145609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01806cb-2128-4950-b35f-c6efab2a7977 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:36.162168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f03e15-0333-483d-87d6-ee467fe731e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:36.163241 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f03e15-0333-483d-87d6-ee467fe731e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:36.164513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f03e15-0333-483d-87d6-ee467fe731e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:36.164513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f03e15-0333-483d-87d6-ee467fe731e0 admin admin] PATH_INFO: `/v3/projects/f1ee747f366a444dba71fd6be7bf4b34/users/f0cfd7d93a6e4c61833a68068d69e61a/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:36.172201 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 809/1632] 10.222.0.93 () {58 vars in 1058 bytes} [Fri Jun 3 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:36.173553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0a9a8-ee2c-4f2b-8e90-3731bcafe8dd None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:36.174736 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0a9a8-ee2c-4f2b-8e90-3731bcafe8dd None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:36.181556 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0a9a8-ee2c-4f2b-8e90-3731bcafe8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:36.209995 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f03e15-0333-483d-87d6-ee467fe731e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ee747f366a444dba71fd6be7bf4b34, user_id=f0cfd7d93a6e4c61833a68068d69e61a, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:36.213708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f03e15-0333-483d-87d6-ee467fe731e0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:36.242123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 824/1633] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:35 2022] PUT /identity/v3/projects/f1ee747f366a444dba71fd6be7bf4b34/users/f0cfd7d93a6e4c61833a68068d69e61a/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:36.243068 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f03e15-0333-483d-87d6-ee467fe731e0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:36.262720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-93b0a9a8-ee2c-4f2b-8e90-3731bcafe8dd None None] MFA Rules not processed for user `28f76dcdfc69420cb20a0ba2f6462449`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:36.263967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-93b0a9a8-ee2c-4f2b-8e90-3731bcafe8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:36.428824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f03e15-0333-483d-87d6-ee467fe731e0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:36.438227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9bb4b-11a2-4c2f-a7f6-4768cd09709a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:36.439369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9bb4b-11a2-4c2f-a7f6-4768cd09709a service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:36.439661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9bb4b-11a2-4c2f-a7f6-4768cd09709a service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:36.439942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9bb4b-11a2-4c2f-a7f6-4768cd09709a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:36.450586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 810/1634] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:36.450586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0a9a8-ee2c-4f2b-8e90-3731bcafe8dd None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:36.580151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9bb4b-11a2-4c2f-a7f6-4768cd09709a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:36.581956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9bb4b-11a2-4c2f-a7f6-4768cd09709a service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:36.602648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0a9a8-ee2c-4f2b-8e90-3731bcafe8dd None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:36.605319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 825/1635] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:36.607271 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711dfde-08df-4487-9e67-280481ac54a2 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:36.607856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711dfde-08df-4487-9e67-280481ac54a2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:36.608376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711dfde-08df-4487-9e67-280481ac54a2 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:36.609436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f7f68-d1ef-495a-a5d1-5050fa290c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:36.610320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7f68-d1ef-495a-a5d1-5050fa290c88 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:36.610917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7f68-d1ef-495a-a5d1-5050fa290c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:36.611305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f7f68-d1ef-495a-a5d1-5050fa290c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:36.612209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f7f68-d1ef-495a-a5d1-5050fa290c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:36.622872 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f7f68-d1ef-495a-a5d1-5050fa290c88 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:36.627909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 826/1636] 10.222.0.93 () {54 vars in 912 bytes} [Fri Jun 3 15:26:36 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:26:36.629615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711dfde-08df-4487-9e67-280481ac54a2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:36.641667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 811/1637] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:36.641667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630589c-e7b0-4cdc-b7c1-bc5753d66d2a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:36.641667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630589c-e7b0-4cdc-b7c1-bc5753d66d2a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:36.641667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630589c-e7b0-4cdc-b7c1-bc5753d66d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:36.671065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-8630589c-e7b0-4cdc-b7c1-bc5753d66d2a None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:36.672521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8630589c-e7b0-4cdc-b7c1-bc5753d66d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:36.690935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 812/1638] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:36.692066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630589c-e7b0-4cdc-b7c1-bc5753d66d2a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:36.715761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630589c-e7b0-4cdc-b7c1-bc5753d66d2a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:36.723343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:36.726797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adde0a-4765-49e9-a4a9-720e47309934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:36.727710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adde0a-4765-49e9-a4a9-720e47309934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:36.728175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adde0a-4765-49e9-a4a9-720e47309934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:36.728488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52adde0a-4765-49e9-a4a9-720e47309934 admin admin] PATH_INFO: `/v3/projects/f1ee747f366a444dba71fd6be7bf4b34/users/f0cfd7d93a6e4c61833a68068d69e61a/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:36.732183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adde0a-4765-49e9-a4a9-720e47309934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ee747f366a444dba71fd6be7bf4b34, user_id=f0cfd7d93a6e4c61833a68068d69e61a, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:36.736168 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52adde0a-4765-49e9-a4a9-720e47309934 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:36.770143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 813/1639] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:36 2022] PUT /identity/v3/projects/f1ee747f366a444dba71fd6be7bf4b34/users/f0cfd7d93a6e4c61833a68068d69e61a/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:36.770143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adde0a-4765-49e9-a4a9-720e47309934 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:36.790065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711dfde-08df-4487-9e67-280481ac54a2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:36.823832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:36.879644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52adde0a-4765-49e9-a4a9-720e47309934 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:36.887693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c0b18-6207-4345-8c8e-4fe2d3e4f39b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:36.888623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c0b18-6207-4345-8c8e-4fe2d3e4f39b service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:36.889314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c0b18-6207-4345-8c8e-4fe2d3e4f39b service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:36.889445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c0b18-6207-4345-8c8e-4fe2d3e4f39b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:36.910964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efb9ac-2448-4921-8460-516bde83af08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:36.911700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efb9ac-2448-4921-8460-516bde83af08 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:36.911918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efb9ac-2448-4921-8460-516bde83af08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:36.912198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efb9ac-2448-4921-8460-516bde83af08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:37.084622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c0b18-6207-4345-8c8e-4fe2d3e4f39b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:37.086436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c0b18-6207-4345-8c8e-4fe2d3e4f39b service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:37.101584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efb9ac-2448-4921-8460-516bde83af08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:37.104563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 814/1640] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:37.106230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efb9ac-2448-4921-8460-516bde83af08 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:37.121354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 827/1641] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:37.123244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf133cd3-ed76-4c5e-bb33-e539b73be2b6 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:37.123702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf133cd3-ed76-4c5e-bb33-e539b73be2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:37.124138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf133cd3-ed76-4c5e-bb33-e539b73be2b6 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:37.126047 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 828/1642] 10.222.0.93 () {54 vars in 912 bytes} [Fri Jun 3 15:26:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:26:37.133663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528862e7-ed96-4414-b35b-3b15c5fb3ae7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:37.133958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528862e7-ed96-4414-b35b-3b15c5fb3ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:37.134361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528862e7-ed96-4414-b35b-3b15c5fb3ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:37.135256 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5542f-7897-4327-9d52-161ba63959ac None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:37.135680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5542f-7897-4327-9d52-161ba63959ac None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:37.135971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5542f-7897-4327-9d52-161ba63959ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:37.181087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-528862e7-ed96-4414-b35b-3b15c5fb3ae7 None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:37.182354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-528862e7-ed96-4414-b35b-3b15c5fb3ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:37.228617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 829/1643] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:37.237705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-06c5542f-7897-4327-9d52-161ba63959ac None None] MFA Rules not processed for user `f0cfd7d93a6e4c61833a68068d69e61a`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:37.238665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-06c5542f-7897-4327-9d52-161ba63959ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:37.241873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528862e7-ed96-4414-b35b-3b15c5fb3ae7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:37.243827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-528862e7-ed96-4414-b35b-3b15c5fb3ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:37.441788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528862e7-ed96-4414-b35b-3b15c5fb3ae7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:37.452918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b717ec-933b-486a-920b-540cea9e9129 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13121a263c31433dbd4a5082b4bf3bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:37.453597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b717ec-933b-486a-920b-540cea9e9129 service glance] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:37.453886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b717ec-933b-486a-920b-540cea9e9129 service glance] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:37.454319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b717ec-933b-486a-920b-540cea9e9129 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:37.465460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 815/1644] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:37.466862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5542f-7897-4327-9d52-161ba63959ac None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:37.479711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b717ec-933b-486a-920b-540cea9e9129 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:37.481603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b717ec-933b-486a-920b-540cea9e9129 service glance] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:37.490240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5542f-7897-4327-9d52-161ba63959ac None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:37.500190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68260d95-2472-44c3-9ac0-e148d57cc9bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:37.502028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68260d95-2472-44c3-9ac0-e148d57cc9bd service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:37.502398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68260d95-2472-44c3-9ac0-e148d57cc9bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:37.504788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68260d95-2472-44c3-9ac0-e148d57cc9bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:37.515475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 830/1645] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:37.640252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b717ec-933b-486a-920b-540cea9e9129 service glance] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:37.658482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b717ec-933b-486a-920b-540cea9e9129 service glance] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:37.663411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:37.668299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68260d95-2472-44c3-9ac0-e148d57cc9bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:37.670305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68260d95-2472-44c3-9ac0-e148d57cc9bd service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:37.670436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab326bc-7f02-46e7-8363-82d4bd9a69ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:37.671137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab326bc-7f02-46e7-8363-82d4bd9a69ff service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:37.671424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab326bc-7f02-46e7-8363-82d4bd9a69ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:37.671706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab326bc-7f02-46e7-8363-82d4bd9a69ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:37.688128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 816/1646] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:37.697943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab326bc-7f02-46e7-8363-82d4bd9a69ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:37.704454 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab326bc-7f02-46e7-8363-82d4bd9a69ff service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:37.723977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 831/1647] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:26:37.860411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68260d95-2472-44c3-9ac0-e148d57cc9bd service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:37.991530 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68260d95-2472-44c3-9ac0-e148d57cc9bd service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:38.009157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcd685-ad25-4618-8b9b-fe264a08cfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:38.011645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcd685-ad25-4618-8b9b-fe264a08cfdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:38.013739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcd685-ad25-4618-8b9b-fe264a08cfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:38.014185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcd685-ad25-4618-8b9b-fe264a08cfdb admin admin] PATH_INFO: `/v3/users/2e3ea44b40d9419aa51e0fa691d8dce9` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:38.017025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcd685-ad25-4618-8b9b-fe264a08cfdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e3ea44b40d9419aa51e0fa691d8dce9)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:38.022575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcd685-ad25-4618-8b9b-fe264a08cfdb admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:38.158777 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c8bcd685-ad25-4618-8b9b-fe264a08cfdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e3ea44b40d9419aa51e0fa691d8dce9'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:38.226076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c8bcd685-ad25-4618-8b9b-fe264a08cfdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e3ea44b40d9419aa51e0fa691d8dce9'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:38.230195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c8bcd685-ad25-4618-8b9b-fe264a08cfdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e3ea44b40d9419aa51e0fa691d8dce9'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:38.260824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c8bcd685-ad25-4618-8b9b-fe264a08cfdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e3ea44b40d9419aa51e0fa691d8dce9'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:38.261691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c8bcd685-ad25-4618-8b9b-fe264a08cfdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e3ea44b40d9419aa51e0fa691d8dce9'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:38.305612 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 817/1648] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:26:37 2022] DELETE /identity/v3/users/2e3ea44b40d9419aa51e0fa691d8dce9 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:38.309018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c2bfc-9dbd-4eca-a9d9-7628db4b5fa5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:38.310330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c2bfc-9dbd-4eca-a9d9-7628db4b5fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:38.310870 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c2bfc-9dbd-4eca-a9d9-7628db4b5fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:38.361884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5e6c2bfc-9dbd-4eca-a9d9-7628db4b5fa5 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:38.363431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5e6c2bfc-9dbd-4eca-a9d9-7628db4b5fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:38.399276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0741895-3593-4cfc-bd4e-498b7406333e None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:38.400232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0741895-3593-4cfc-bd4e-498b7406333e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:38.401345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0741895-3593-4cfc-bd4e-498b7406333e None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:38.403700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:38.405834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 818/1649] 10.222.0.93 () {54 vars in 912 bytes} [Fri Jun 3 15:26:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:26:38.416915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ee62-3acb-45e4-a443-3d43338fa16d None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:38.418177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ee62-3acb-45e4-a443-3d43338fa16d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:38.418863 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986ee62-3acb-45e4-a443-3d43338fa16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:38.484768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 832/1650] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:38.547415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d986ee62-3acb-45e4-a443-3d43338fa16d None None] MFA Rules not processed for user `e3f887d5bbcd424884537b95affd96f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:38.550721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c2bfc-9dbd-4eca-a9d9-7628db4b5fa5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:38.552682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5e6c2bfc-9dbd-4eca-a9d9-7628db4b5fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:38.553483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d986ee62-3acb-45e4-a443-3d43338fa16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:38.700723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 819/1651] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:38.708565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986ee62-3acb-45e4-a443-3d43338fa16d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:38.710439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d986ee62-3acb-45e4-a443-3d43338fa16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:38.715598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c2bfc-9dbd-4eca-a9d9-7628db4b5fa5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:38.730115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986ee62-3acb-45e4-a443-3d43338fa16d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:38.730311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e2cc9-90e0-4722-afbc-ac383fc0ca61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:38.730985 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e2cc9-90e0-4722-afbc-ac383fc0ca61 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:38.731273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e2cc9-90e0-4722-afbc-ac383fc0ca61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:38.731550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e2cc9-90e0-4722-afbc-ac383fc0ca61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:38.741282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:38.742764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:38.743307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:38.743975 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] PATH_INFO: `/v3/users/9d90b1fcab1c4477b4b2d6c6c219c91b` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:38.745672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d90b1fcab1c4477b4b2d6c6c219c91b)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:38.748935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:38.751937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e2cc9-90e0-4722-afbc-ac383fc0ca61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:38.753877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e2cc9-90e0-4722-afbc-ac383fc0ca61 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:38.754455 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:38.779554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 833/1652] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:38.780446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e2cc9-90e0-4722-afbc-ac383fc0ca61 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:38.781850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7b2e2cc9-90e0-4722-afbc-ac383fc0ca61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:38.795581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e2cc9-90e0-4722-afbc-ac383fc0ca61 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:38.801320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00083af4-cccf-498c-b16e-2c3dbc6bfa2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:38.802589 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00083af4-cccf-498c-b16e-2c3dbc6bfa2a service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:38.803134 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00083af4-cccf-498c-b16e-2c3dbc6bfa2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:38.803639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00083af4-cccf-498c-b16e-2c3dbc6bfa2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:38.814414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00083af4-cccf-498c-b16e-2c3dbc6bfa2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:38.815940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00083af4-cccf-498c-b16e-2c3dbc6bfa2a service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:38.827123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d90b1fcab1c4477b4b2d6c6c219c91b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:38.828170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 834/1653] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:38.844323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d90b1fcab1c4477b4b2d6c6c219c91b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:38.845095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d90b1fcab1c4477b4b2d6c6c219c91b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:38.857180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d90b1fcab1c4477b4b2d6c6c219c91b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:38.858300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d90b1fcab1c4477b4b2d6c6c219c91b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:38.878518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 820/1654] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:26:38 2022] DELETE /identity/v3/users/9d90b1fcab1c4477b4b2d6c6c219c91b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:38.882680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0359-9004-4dc0-a8c2-dee390bfcda9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:38.883051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0359-9004-4dc0-a8c2-dee390bfcda9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:38.883424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0359-9004-4dc0-a8c2-dee390bfcda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:38.910205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c2cd0359-9004-4dc0-a8c2-dee390bfcda9 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:38.912233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c2cd0359-9004-4dc0-a8c2-dee390bfcda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:39.040787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 835/1655] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:26:39.082334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:39.084694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:39.198039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86191951-5287-4fad-8de3-63d2a16a0dbc admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:39.203209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6cd94-bd3a-46c0-bfc5-8174c779570d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:39.203884 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6cd94-bd3a-46c0-bfc5-8174c779570d service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:39.204164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6cd94-bd3a-46c0-bfc5-8174c779570d service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:39.204438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6cd94-bd3a-46c0-bfc5-8174c779570d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:39.229494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6cd94-bd3a-46c0-bfc5-8174c779570d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:39.231361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6cd94-bd3a-46c0-bfc5-8174c779570d service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:39.246433 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 821/1656] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:39.903727 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6259-9c19-4015-8b0b-ae173af57cc3 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:39.904159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6259-9c19-4015-8b0b-ae173af57cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:39.904881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d6259-9c19-4015-8b0b-ae173af57cc3 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:39.906773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 836/1657] 10.222.0.93 () {54 vars in 960 bytes} [Fri Jun 3 15:26:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:26:39.914751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30c663-ee78-4904-bdf7-3f10650292f1 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:39.916843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30c663-ee78-4904-bdf7-3f10650292f1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:39.917285 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30c663-ee78-4904-bdf7-3f10650292f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:39.960840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-eb30c663-ee78-4904-bdf7-3f10650292f1 None None] MFA Rules not processed for user `50625c2b38ed41698ba6cccdb12730ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:39.962754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-eb30c663-ee78-4904-bdf7-3f10650292f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:40.136217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 822/1658] 10.222.0.93 () {58 vars in 1057 bytes} [Fri Jun 3 15:26:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:40.143881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6259-9c19-4015-8b0b-ae173af57cc3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:40.174892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d6259-9c19-4015-8b0b-ae173af57cc3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:40.182746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27bbc4-888a-4b38-8502-74a660212a47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:40.184588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27bbc4-888a-4b38-8502-74a660212a47 service placement] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:40.187071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27bbc4-888a-4b38-8502-74a660212a47 service placement] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:40.187684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27bbc4-888a-4b38-8502-74a660212a47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:40.190577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4d27bbc4-888a-4b38-8502-74a660212a47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:40.222231 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27bbc4-888a-4b38-8502-74a660212a47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:40.224545 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27bbc4-888a-4b38-8502-74a660212a47 service placement] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:40.244949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 837/1659] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:41.137411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30c663-ee78-4904-bdf7-3f10650292f1 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:41.150143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-eb30c663-ee78-4904-bdf7-3f10650292f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:41.212053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb30c663-ee78-4904-bdf7-3f10650292f1 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:41.230740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:41.236215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:41.236846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:41.237330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] PATH_INFO: `/v3/projects/b710b999eb884a96a85e9c6debe49306` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:41.254754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b710b999eb884a96a85e9c6debe49306)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:41.267007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: WARNING py.warnings [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:26:41.267007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: warnings.warn(msg) Jun 03 15:26:41.267007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: Jun 03 15:26:41.272335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:41.332777 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:41.458688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b710b999eb884a96a85e9c6debe49306', 'request_id': 'req-79d78ad9-8b80-45f8-968a-a4a028478e6c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:41.498902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b710b999eb884a96a85e9c6debe49306', 'request_id': 'req-79d78ad9-8b80-45f8-968a-a4a028478e6c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:41.542827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b710b999eb884a96a85e9c6debe49306', 'request_id': 'req-79d78ad9-8b80-45f8-968a-a4a028478e6c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:41.600163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27bbc4-888a-4b38-8502-74a660212a47 service placement] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:41.605125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] The token cache is being invalidate because project b710b999eb884a96a85e9c6debe49306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:26:41.605125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:41.605754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:41.607587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4d27bbc4-888a-4b38-8502-74a660212a47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:41.618583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 823/1660] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:26:41 2022] DELETE /identity/v3/projects/b710b999eb884a96a85e9c6debe49306 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:41.618583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:41.618583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:41.643249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27bbc4-888a-4b38-8502-74a660212a47 service placement] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:41.752110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:41.813057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:41.817413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d78ad9-8b80-45f8-968a-a4a028478e6c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:41.817613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:41.817613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:41.817613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] PATH_INFO: `/v3/projects/8e608c77c1ac4f4d9a62d95d45ad9d5a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:41.819348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e608c77c1ac4f4d9a62d95d45ad9d5a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:41.827036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:41.829066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c41ab-7bff-4042-851a-bdd6dc654113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:41.830161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c41ab-7bff-4042-851a-bdd6dc654113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:41.830444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c41ab-7bff-4042-851a-bdd6dc654113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:41.830747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c41ab-7bff-4042-851a-bdd6dc654113 admin admin] PATH_INFO: `/v3/users/4de5b16e95b542428d827bba01fcc377` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:41.835215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c41ab-7bff-4042-851a-bdd6dc654113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4de5b16e95b542428d827bba01fcc377)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:41.839562 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c41ab-7bff-4042-851a-bdd6dc654113 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:41.969659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-469c41ab-7bff-4042-851a-bdd6dc654113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4de5b16e95b542428d827bba01fcc377'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:41.993524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e608c77c1ac4f4d9a62d95d45ad9d5a', 'request_id': 'req-9d01b1ab-56eb-412c-b50a-07534965e71b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:41.999033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-469c41ab-7bff-4042-851a-bdd6dc654113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4de5b16e95b542428d827bba01fcc377'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:41.999801 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-469c41ab-7bff-4042-851a-bdd6dc654113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4de5b16e95b542428d827bba01fcc377'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.017019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e608c77c1ac4f4d9a62d95d45ad9d5a', 'request_id': 'req-9d01b1ab-56eb-412c-b50a-07534965e71b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.021214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-469c41ab-7bff-4042-851a-bdd6dc654113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4de5b16e95b542428d827bba01fcc377'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.022000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-469c41ab-7bff-4042-851a-bdd6dc654113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4de5b16e95b542428d827bba01fcc377'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.039340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e608c77c1ac4f4d9a62d95d45ad9d5a', 'request_id': 'req-9d01b1ab-56eb-412c-b50a-07534965e71b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.063659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 824/1661] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:26:41 2022] DELETE /identity/v3/users/4de5b16e95b542428d827bba01fcc377 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:42.065457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fccdc-d5b5-440d-9fb5-532c1371bd26 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:42.065756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fccdc-d5b5-440d-9fb5-532c1371bd26 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:42.066035 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fccdc-d5b5-440d-9fb5-532c1371bd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:42.081704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] The token cache is being invalidate because project 8e608c77c1ac4f4d9a62d95d45ad9d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:26:42.082379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.085193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d01b1ab-56eb-412c-b50a-07534965e71b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.093328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 838/1662] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:26:41 2022] DELETE /identity/v3/projects/8e608c77c1ac4f4d9a62d95d45ad9d5a => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:42.100722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69b665-6337-45da-b601-d61bac8c56f1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:42.101399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69b665-6337-45da-b601-d61bac8c56f1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:42.101817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69b665-6337-45da-b601-d61bac8c56f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:42.119406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c17fccdc-d5b5-440d-9fb5-532c1371bd26 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:42.120683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c17fccdc-d5b5-440d-9fb5-532c1371bd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:42.155057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:42.169863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ac69b665-6337-45da-b601-d61bac8c56f1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:42.171571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ac69b665-6337-45da-b601-d61bac8c56f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:42.287030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 825/1663] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:42.288075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fccdc-d5b5-440d-9fb5-532c1371bd26 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:42.289547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c17fccdc-d5b5-440d-9fb5-532c1371bd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:42.334485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fccdc-d5b5-440d-9fb5-532c1371bd26 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:42.358916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362da9e3-3508-4e38-9764-93b0eb57fde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:42.361784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362da9e3-3508-4e38-9764-93b0eb57fde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:42.362460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362da9e3-3508-4e38-9764-93b0eb57fde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:42.363384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362da9e3-3508-4e38-9764-93b0eb57fde7 admin admin] PATH_INFO: `/v3/users/f0cfd7d93a6e4c61833a68068d69e61a` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:42.365004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362da9e3-3508-4e38-9764-93b0eb57fde7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cfd7d93a6e4c61833a68068d69e61a)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:42.371520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362da9e3-3508-4e38-9764-93b0eb57fde7 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:42.434827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 839/1664] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:42.436378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386164-fe83-4e69-b3cc-7d664612b3ed None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:42.436668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386164-fe83-4e69-b3cc-7d664612b3ed None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:42.436947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35386164-fe83-4e69-b3cc-7d664612b3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:42.554639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-35386164-fe83-4e69-b3cc-7d664612b3ed None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:42.556587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-35386164-fe83-4e69-b3cc-7d664612b3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:42.557943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-362da9e3-3508-4e38-9764-93b0eb57fde7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cfd7d93a6e4c61833a68068d69e61a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.591023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 840/1665] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:42.591904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35386164-fe83-4e69-b3cc-7d664612b3ed None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:42.596559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-35386164-fe83-4e69-b3cc-7d664612b3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:42.597934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-362da9e3-3508-4e38-9764-93b0eb57fde7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cfd7d93a6e4c61833a68068d69e61a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.619712 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-362da9e3-3508-4e38-9764-93b0eb57fde7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cfd7d93a6e4c61833a68068d69e61a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.677979 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-362da9e3-3508-4e38-9764-93b0eb57fde7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cfd7d93a6e4c61833a68068d69e61a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.685464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-362da9e3-3508-4e38-9764-93b0eb57fde7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cfd7d93a6e4c61833a68068d69e61a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:42.752865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 826/1666] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:26:42 2022] DELETE /identity/v3/users/f0cfd7d93a6e4c61833a68068d69e61a => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:42.759338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff5634-dc4b-4122-9bc6-379925b78f9b None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:42.761147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff5634-dc4b-4122-9bc6-379925b78f9b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:42.761465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff5634-dc4b-4122-9bc6-379925b78f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:42.865768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-07ff5634-dc4b-4122-9bc6-379925b78f9b None None] MFA Rules not processed for user `8632496bdc1b4d06a74caf483c8b0a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:42.868098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-07ff5634-dc4b-4122-9bc6-379925b78f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:42.904284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35386164-fe83-4e69-b3cc-7d664612b3ed None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:42.963058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:43.071188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeed655-ef53-4d5a-88a1-f849a402d2e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:43.072450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeed655-ef53-4d5a-88a1-f849a402d2e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:43.072742 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeed655-ef53-4d5a-88a1-f849a402d2e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:43.073023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeed655-ef53-4d5a-88a1-f849a402d2e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:43.076047 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-baeed655-ef53-4d5a-88a1-f849a402d2e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:43.091234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 827/1667] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:43.093755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff5634-dc4b-4122-9bc6-379925b78f9b None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:43.100549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-07ff5634-dc4b-4122-9bc6-379925b78f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:43.199112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeed655-ef53-4d5a-88a1-f849a402d2e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:43.201854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeed655-ef53-4d5a-88a1-f849a402d2e3 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:43.222413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 841/1668] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:43.223351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeed655-ef53-4d5a-88a1-f849a402d2e3 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:43.231840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-baeed655-ef53-4d5a-88a1-f849a402d2e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:43.248747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:43.411720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff5634-dc4b-4122-9bc6-379925b78f9b None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:43.425673 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f4ff9-ea05-4831-ad51-a5eadb13eff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:43.426305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f4ff9-ea05-4831-ad51-a5eadb13eff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:43.427058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f4ff9-ea05-4831-ad51-a5eadb13eff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:43.427058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f4ff9-ea05-4831-ad51-a5eadb13eff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:43.427734 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f4ff9-ea05-4831-ad51-a5eadb13eff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:43.428045 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f4ff9-ea05-4831-ad51-a5eadb13eff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:43.429835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f4ff9-ea05-4831-ad51-a5eadb13eff7 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:43.461418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeed655-ef53-4d5a-88a1-f849a402d2e3 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:43.473118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 828/1669] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:26:43.477000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f4ff9-ea05-4831-ad51-a5eadb13eff7 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:43.486636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76b5d8-16c7-4572-8341-e9d70e58cf43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:43.487730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76b5d8-16c7-4572-8341-e9d70e58cf43 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:43.490448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76b5d8-16c7-4572-8341-e9d70e58cf43 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:43.491018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76b5d8-16c7-4572-8341-e9d70e58cf43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:43.500420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f4ff9-ea05-4831-ad51-a5eadb13eff7 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:43.517896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1d71f-f257-425c-aba2-0e9ae5df108d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:43.518759 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d71f-f257-425c-aba2-0e9ae5df108d admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:43.519046 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d71f-f257-425c-aba2-0e9ae5df108d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:43.519326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1d71f-f257-425c-aba2-0e9ae5df108d admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:43.520115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1d71f-f257-425c-aba2-0e9ae5df108d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:43.525251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1d71f-f257-425c-aba2-0e9ae5df108d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:43.546282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76b5d8-16c7-4572-8341-e9d70e58cf43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:43.552904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76b5d8-16c7-4572-8341-e9d70e58cf43 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:43.590428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 842/1670] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:43.602872 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 829/1671] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:43 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:26:43.606775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76b5d8-16c7-4572-8341-e9d70e58cf43 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:43.641769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4b327-4d26-456a-91f6-93e7c2a453dd None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:43.642910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4b327-4d26-456a-91f6-93e7c2a453dd None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:43.643386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4b327-4d26-456a-91f6-93e7c2a453dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:43.650188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76b5d8-16c7-4572-8341-e9d70e58cf43 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:43.686406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-79b4b327-4d26-456a-91f6-93e7c2a453dd None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:43.689998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-79b4b327-4d26-456a-91f6-93e7c2a453dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:43.850202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:43.881272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72a465-dcca-410e-8ebf-b8f0c80d0920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:43.883076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72a465-dcca-410e-8ebf-b8f0c80d0920 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:43.885432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72a465-dcca-410e-8ebf-b8f0c80d0920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:43.885768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72a465-dcca-410e-8ebf-b8f0c80d0920 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:43.886742 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72a465-dcca-410e-8ebf-b8f0c80d0920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:43.891134 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72a465-dcca-410e-8ebf-b8f0c80d0920 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:43.906139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 830/1672] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:26:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:43.957726 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 843/1673] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:43 2022] POST /identity/v3/users => generated 294 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:43.960077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4b327-4d26-456a-91f6-93e7c2a453dd None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:43.982525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4b327-4d26-456a-91f6-93e7c2a453dd None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:43.992554 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b391d54-729c-4fa3-98b9-94e8792bcdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:43.994500 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b391d54-729c-4fa3-98b9-94e8792bcdfe admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:43.994849 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b391d54-729c-4fa3-98b9-94e8792bcdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:43.995270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b391d54-729c-4fa3-98b9-94e8792bcdfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:43.997740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b391d54-729c-4fa3-98b9-94e8792bcdfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:44.004185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b391d54-729c-4fa3-98b9-94e8792bcdfe admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:44.039532 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 831/1674] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:44.044109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72a465-dcca-410e-8ebf-b8f0c80d0920 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:44.059495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72a465-dcca-410e-8ebf-b8f0c80d0920 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:44.068160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954d82c-62c0-4535-b2c4-8ef910c35c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:44.069203 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954d82c-62c0-4535-b2c4-8ef910c35c56 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:44.069673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954d82c-62c0-4535-b2c4-8ef910c35c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:44.070082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954d82c-62c0-4535-b2c4-8ef910c35c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:44.072430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954d82c-62c0-4535-b2c4-8ef910c35c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:44.074320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954d82c-62c0-4535-b2c4-8ef910c35c56 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:44.087567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 844/1675] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:44.091649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b391d54-729c-4fa3-98b9-94e8792bcdfe admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:44.105235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b391d54-729c-4fa3-98b9-94e8792bcdfe admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:44.119689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad6a84-a2fc-4780-84ec-0f0817055c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:44.121223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad6a84-a2fc-4780-84ec-0f0817055c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:44.121873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad6a84-a2fc-4780-84ec-0f0817055c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:44.122748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad6a84-a2fc-4780-84ec-0f0817055c92 admin admin] PATH_INFO: `/v3/projects/3aa3fbd5dc7644079c9a3a5419b52d68/users/319ff6b0e6a843d788bfda1949c4bf8a/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:44.163926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad6a84-a2fc-4780-84ec-0f0817055c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3aa3fbd5dc7644079c9a3a5419b52d68, user_id=319ff6b0e6a843d788bfda1949c4bf8a, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:44.168297 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad6a84-a2fc-4780-84ec-0f0817055c92 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:44.198549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 832/1676] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:44 2022] PUT /identity/v3/projects/3aa3fbd5dc7644079c9a3a5419b52d68/users/319ff6b0e6a843d788bfda1949c4bf8a/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:26:44.210218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a997a-f70c-4f6c-8ab6-a19008362393 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:44.210466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a997a-f70c-4f6c-8ab6-a19008362393 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:44.210791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a997a-f70c-4f6c-8ab6-a19008362393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:44.267562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-fe4a997a-f70c-4f6c-8ab6-a19008362393 None None] MFA Rules not processed for user `319ff6b0e6a843d788bfda1949c4bf8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:44.268813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fe4a997a-f70c-4f6c-8ab6-a19008362393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:44.272674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad6a84-a2fc-4780-84ec-0f0817055c92 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:44.275442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-40ad6a84-a2fc-4780-84ec-0f0817055c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:44.475453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad6a84-a2fc-4780-84ec-0f0817055c92 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:44.475639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 845/1677] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:44.476479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a997a-f70c-4f6c-8ab6-a19008362393 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:44.485850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe9404-f329-4ef1-a4f0-f3071ef933f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:44.486685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9404-f329-4ef1-a4f0-f3071ef933f2 service swift] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:44.487082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9404-f329-4ef1-a4f0-f3071ef933f2 service swift] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:44.487487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe9404-f329-4ef1-a4f0-f3071ef933f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:44.501333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:44.661851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe9404-f329-4ef1-a4f0-f3071ef933f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:44.661851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe9404-f329-4ef1-a4f0-f3071ef933f2 service swift] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:44.669053 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a997a-f70c-4f6c-8ab6-a19008362393 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:44.689203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 833/1678] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:26:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:26:44.693225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad66335-3090-43c0-8026-54a67fb14dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:44.694335 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66335-3090-43c0-8026-54a67fb14dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:44.694815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66335-3090-43c0-8026-54a67fb14dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:44.695462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66335-3090-43c0-8026-54a67fb14dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:44.696397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad66335-3090-43c0-8026-54a67fb14dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:44.699045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad66335-3090-43c0-8026-54a67fb14dd4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:44.770323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 846/1679] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:44 2022] POST /identity/v3/projects => generated 387 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:44.774153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe9404-f329-4ef1-a4f0-f3071ef933f2 service swift] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:44.790809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe9404-f329-4ef1-a4f0-f3071ef933f2 service swift] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:45.036370 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58e6e7-c5af-467c-81c9-b1ca31a66cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:45.039947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58e6e7-c5af-467c-81c9-b1ca31a66cfe admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:45.040419 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58e6e7-c5af-467c-81c9-b1ca31a66cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:45.040810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b58e6e7-c5af-467c-81c9-b1ca31a66cfe admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:45.041655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58e6e7-c5af-467c-81c9-b1ca31a66cfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:45.047581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b58e6e7-c5af-467c-81c9-b1ca31a66cfe admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:45.127372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 834/1680] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:44 2022] POST /identity/v3/users => generated 300 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:45.131992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad66335-3090-43c0-8026-54a67fb14dd4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:45.159258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad66335-3090-43c0-8026-54a67fb14dd4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:45.194115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fd573-693f-45d6-be1c-3f18fdc7f267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:45.194832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd573-693f-45d6-be1c-3f18fdc7f267 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:45.195127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd573-693f-45d6-be1c-3f18fdc7f267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:45.195412 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd573-693f-45d6-be1c-3f18fdc7f267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:45.196163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fd573-693f-45d6-be1c-3f18fdc7f267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:45.209487 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fd573-693f-45d6-be1c-3f18fdc7f267 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:45.257348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58e6e7-c5af-467c-81c9-b1ca31a66cfe admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:45.270611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 847/1681] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:45.276680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b58e6e7-c5af-467c-81c9-b1ca31a66cfe admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:45.301971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b1612-45c6-41b8-a9ba-44df2a06800a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:45.312356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b1612-45c6-41b8-a9ba-44df2a06800a admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:45.312356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b1612-45c6-41b8-a9ba-44df2a06800a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:45.312356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b1612-45c6-41b8-a9ba-44df2a06800a admin admin] PATH_INFO: `/v3/projects/26306aaa359c45ceaad844030abc3403/users/757b0aa463f848e6aaacb1d958dc34c0/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:45.361839 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b1612-45c6-41b8-a9ba-44df2a06800a admin admin] RBAC: Authorizing `identity:create_grant(project_id=26306aaa359c45ceaad844030abc3403, user_id=757b0aa463f848e6aaacb1d958dc34c0, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:45.373715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b1612-45c6-41b8-a9ba-44df2a06800a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:45.423578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 835/1682] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:45 2022] PUT /identity/v3/projects/26306aaa359c45ceaad844030abc3403/users/757b0aa463f848e6aaacb1d958dc34c0/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:26:45.424806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fd573-693f-45d6-be1c-3f18fdc7f267 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:45.439289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fd573-693f-45d6-be1c-3f18fdc7f267 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:45.615060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:45.740323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0d3b4-f445-4350-b9b4-6f188c004a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:45.741426 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0d3b4-f445-4350-b9b4-6f188c004a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:45.741886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0d3b4-f445-4350-b9b4-6f188c004a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:45.742308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0d3b4-f445-4350-b9b4-6f188c004a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:45.746368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0d3b4-f445-4350-b9b4-6f188c004a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:45.748424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0d3b4-f445-4350-b9b4-6f188c004a3a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:45.778929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 848/1683] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:45.784720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b1612-45c6-41b8-a9ba-44df2a06800a admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:45.803637 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b1612-45c6-41b8-a9ba-44df2a06800a admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:45.832942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8189d1-a95d-4cb6-87fe-e842ba6f2a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:45.835100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8189d1-a95d-4cb6-87fe-e842ba6f2a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:45.835635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8189d1-a95d-4cb6-87fe-e842ba6f2a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:45.836140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8189d1-a95d-4cb6-87fe-e842ba6f2a57 admin admin] PATH_INFO: `/v3/projects/26306aaa359c45ceaad844030abc3403/users/757b0aa463f848e6aaacb1d958dc34c0/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:45.841793 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8189d1-a95d-4cb6-87fe-e842ba6f2a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26306aaa359c45ceaad844030abc3403, user_id=757b0aa463f848e6aaacb1d958dc34c0, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:45.854815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8189d1-a95d-4cb6-87fe-e842ba6f2a57 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:45.911705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 836/1684] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:45 2022] PUT /identity/v3/projects/26306aaa359c45ceaad844030abc3403/users/757b0aa463f848e6aaacb1d958dc34c0/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:45.925197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8727021-c2ba-4538-9006-7172f90c0cb1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:45.925637 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8727021-c2ba-4538-9006-7172f90c0cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:45.925948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8727021-c2ba-4538-9006-7172f90c0cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:45.969469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c8727021-c2ba-4538-9006-7172f90c0cb1 None None] MFA Rules not processed for user `757b0aa463f848e6aaacb1d958dc34c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:45.971144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c8727021-c2ba-4538-9006-7172f90c0cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:46.096917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 849/1685] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:46.119583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8189d1-a95d-4cb6-87fe-e842ba6f2a57 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:46.137557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9f8189d1-a95d-4cb6-87fe-e842ba6f2a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:46.313429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8189d1-a95d-4cb6-87fe-e842ba6f2a57 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:46.326931 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37601d9-1bc9-4412-9935-079803fb21f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:46.327836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37601d9-1bc9-4412-9935-079803fb21f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:46.328856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37601d9-1bc9-4412-9935-079803fb21f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:46.329273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37601d9-1bc9-4412-9935-079803fb21f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:46.350484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37601d9-1bc9-4412-9935-079803fb21f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:46.358830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37601d9-1bc9-4412-9935-079803fb21f2 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:46.380016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 837/1686] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:46.448909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8727021-c2ba-4538-9006-7172f90c0cb1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:46.607245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8727021-c2ba-4538-9006-7172f90c0cb1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:46.619881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:46.620826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:46.621133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:46.621568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] PATH_INFO: `/v3/users/319ff6b0e6a843d788bfda1949c4bf8a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:46.626358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] RBAC: Authorizing `identity:delete_user(user_id=319ff6b0e6a843d788bfda1949c4bf8a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:46.627717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: WARNING py.warnings [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 03 15:26:46.627717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: warnings.warn(msg) Jun 03 15:26:46.627717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: Jun 03 15:26:46.636086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:46.808851 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '319ff6b0e6a843d788bfda1949c4bf8a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:46.838858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:46.858008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '319ff6b0e6a843d788bfda1949c4bf8a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:46.859060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '319ff6b0e6a843d788bfda1949c4bf8a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:46.881557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '319ff6b0e6a843d788bfda1949c4bf8a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:46.882452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '319ff6b0e6a843d788bfda1949c4bf8a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:46.907420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 850/1687] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:26:46 2022] DELETE /identity/v3/users/319ff6b0e6a843d788bfda1949c4bf8a => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:46.912047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5638299-956f-42dd-be18-b23ca905f910 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:46.912589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5638299-956f-42dd-be18-b23ca905f910 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:46.912980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5638299-956f-42dd-be18-b23ca905f910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:46.919230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:46.950824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c5638299-956f-42dd-be18-b23ca905f910 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:46.953432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c5638299-956f-42dd-be18-b23ca905f910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:47.052066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 838/1688] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:47.073365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:47.075936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:47.270886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09448ad8-470a-4108-9572-bbacef27e69a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:47.284887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a36cd6-77b7-4010-b4a6-4e2e283d9128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:47.286419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36cd6-77b7-4010-b4a6-4e2e283d9128 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:47.287257 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36cd6-77b7-4010-b4a6-4e2e283d9128 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:47.287813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a36cd6-77b7-4010-b4a6-4e2e283d9128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:47.309552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a36cd6-77b7-4010-b4a6-4e2e283d9128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:47.311746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a36cd6-77b7-4010-b4a6-4e2e283d9128 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:47.345342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 851/1689] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:47.636978 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5638299-956f-42dd-be18-b23ca905f910 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:47.638592 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c5638299-956f-42dd-be18-b23ca905f910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:47.672539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5638299-956f-42dd-be18-b23ca905f910 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:47.691429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:47.692725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:47.693223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:47.693725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] PATH_INFO: `/v3/projects/03e032dba06e4185b897268d78a35290` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:47.695227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e032dba06e4185b897268d78a35290)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:47.698215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:47.753965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a36cd6-77b7-4010-b4a6-4e2e283d9128 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:47.756231 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-11a36cd6-77b7-4010-b4a6-4e2e283d9128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:47.788833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a36cd6-77b7-4010-b4a6-4e2e283d9128 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:47.796857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e032dba06e4185b897268d78a35290', 'request_id': 'req-59af964e-b7fe-4e82-a79e-63c64e299e97'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:47.824010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e032dba06e4185b897268d78a35290', 'request_id': 'req-59af964e-b7fe-4e82-a79e-63c64e299e97'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:47.847366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e032dba06e4185b897268d78a35290', 'request_id': 'req-59af964e-b7fe-4e82-a79e-63c64e299e97'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:47.868068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] The token cache is being invalidate because project 03e032dba06e4185b897268d78a35290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:26:47.869749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:47.870608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:47.872950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 839/1690] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:26:47 2022] DELETE /identity/v3/projects/03e032dba06e4185b897268d78a35290 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:47.877474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:47.879381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:47.899513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:47.900588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:47.904371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:47.904826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] PATH_INFO: `/v3/projects/f1ee747f366a444dba71fd6be7bf4b34` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:47.909337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ee747f366a444dba71fd6be7bf4b34)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:47.911647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:48.012943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ee747f366a444dba71fd6be7bf4b34', 'request_id': 'req-befbad0e-7d86-482b-aa33-4a6e5253a985'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:48.025031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ee747f366a444dba71fd6be7bf4b34', 'request_id': 'req-befbad0e-7d86-482b-aa33-4a6e5253a985'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:48.058142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ee747f366a444dba71fd6be7bf4b34', 'request_id': 'req-befbad0e-7d86-482b-aa33-4a6e5253a985'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:48.082397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] The token cache is being invalidate because project f1ee747f366a444dba71fd6be7bf4b34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:26:48.083129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:48.084529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-befbad0e-7d86-482b-aa33-4a6e5253a985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:48.092825 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 852/1691] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:26:47 2022] DELETE /identity/v3/projects/f1ee747f366a444dba71fd6be7bf4b34 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:26:48.094525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961566d-0632-4309-af13-d837603b449d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:48.094889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961566d-0632-4309-af13-d837603b449d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:48.095141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2961566d-0632-4309-af13-d837603b449d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:48.115812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af964e-b7fe-4e82-a79e-63c64e299e97 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:48.121734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2961566d-0632-4309-af13-d837603b449d None None] MFA Rules not processed for user `e3f887d5bbcd424884537b95affd96f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:48.126472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2961566d-0632-4309-af13-d837603b449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:48.131561 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:48.223804 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:48.304928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 853/1692] 10.222.0.95 () {58 vars in 1015 bytes} [Fri Jun 3 15:26:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:48.308644 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19229c15-7740-48ca-9985-3592a0e95287 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:48.308943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19229c15-7740-48ca-9985-3592a0e95287 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:48.309229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19229c15-7740-48ca-9985-3592a0e95287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:48.333640 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:48.337642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:48.337987 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:48.338480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] PATH_INFO: `/v3/users/28f76dcdfc69420cb20a0ba2f6462449` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:48.348862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f76dcdfc69420cb20a0ba2f6462449)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:48.354452 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:48.375592 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-19229c15-7740-48ca-9985-3592a0e95287 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:48.380087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-19229c15-7740-48ca-9985-3592a0e95287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:48.425325 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 854/1693] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:48.428623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dc972-bd55-4a29-806c-6d9230099ad7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:48.428946 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dc972-bd55-4a29-806c-6d9230099ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:48.429230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dc972-bd55-4a29-806c-6d9230099ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:48.496535 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7b0dc972-bd55-4a29-806c-6d9230099ad7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:48.497948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7b0dc972-bd55-4a29-806c-6d9230099ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:48.506973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f76dcdfc69420cb20a0ba2f6462449'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:48.535385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 855/1694] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:48.549093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dc972-bd55-4a29-806c-6d9230099ad7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:48.553287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f76dcdfc69420cb20a0ba2f6462449'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:48.555117 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f76dcdfc69420cb20a0ba2f6462449'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:48.573385 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f76dcdfc69420cb20a0ba2f6462449'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:48.579067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f76dcdfc69420cb20a0ba2f6462449'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:48.581861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7b0dc972-bd55-4a29-806c-6d9230099ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:48.611707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dc972-bd55-4a29-806c-6d9230099ad7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:48.611915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 840/1695] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:26:47 2022] DELETE /identity/v3/users/28f76dcdfc69420cb20a0ba2f6462449 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:26:48.612736 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:48.624751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:48.707000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf023e98-5b7a-4acb-9720-a4c395b22aed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:48.709173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf023e98-5b7a-4acb-9720-a4c395b22aed service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:48.710793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf023e98-5b7a-4acb-9720-a4c395b22aed service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:48.714687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf023e98-5b7a-4acb-9720-a4c395b22aed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:48.721423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cf023e98-5b7a-4acb-9720-a4c395b22aed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:48.745482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf023e98-5b7a-4acb-9720-a4c395b22aed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:48.748370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf023e98-5b7a-4acb-9720-a4c395b22aed service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:48.775155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 856/1696] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:48.791730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5590b2-9d33-46ad-986d-134c8d009cbf admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:48.800628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231b6b0-3760-49d0-8914-c1abf3324119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:48.801844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231b6b0-3760-49d0-8914-c1abf3324119 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:48.802418 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231b6b0-3760-49d0-8914-c1abf3324119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:48.803000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231b6b0-3760-49d0-8914-c1abf3324119 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:48.803993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231b6b0-3760-49d0-8914-c1abf3324119 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:48.804551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231b6b0-3760-49d0-8914-c1abf3324119 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:48.807233 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231b6b0-3760-49d0-8914-c1abf3324119 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:48.847595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 841/1697] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:26:48.852832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf023e98-5b7a-4acb-9720-a4c395b22aed service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:48.869607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf023e98-5b7a-4acb-9720-a4c395b22aed service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:48.876135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcb988-ebf9-4f2f-bb98-16e7cd95faf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:48.877533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcb988-ebf9-4f2f-bb98-16e7cd95faf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:48.878332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcb988-ebf9-4f2f-bb98-16e7cd95faf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:48.879110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebcb988-ebf9-4f2f-bb98-16e7cd95faf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:48.880335 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcb988-ebf9-4f2f-bb98-16e7cd95faf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:48.883403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebcb988-ebf9-4f2f-bb98-16e7cd95faf0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:48.951102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 857/1698] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:48 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:26:48.955227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231b6b0-3760-49d0-8914-c1abf3324119 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:48.972996 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231b6b0-3760-49d0-8914-c1abf3324119 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:49.136536 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:49.181618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb551f3b-473d-437f-be17-21569ac7227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:49.183105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb551f3b-473d-437f-be17-21569ac7227c admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:49.185357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb551f3b-473d-437f-be17-21569ac7227c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:49.185748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb551f3b-473d-437f-be17-21569ac7227c admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:49.186652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb551f3b-473d-437f-be17-21569ac7227c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:49.192169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb551f3b-473d-437f-be17-21569ac7227c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:49.262519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcb988-ebf9-4f2f-bb98-16e7cd95faf0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:49.263158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 842/1699] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:48 2022] POST /identity/v3/users => generated 289 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:49.278557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebcb988-ebf9-4f2f-bb98-16e7cd95faf0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:49.292563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a47f13-1af9-412e-889b-7efa2059f13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:49.293583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a47f13-1af9-412e-889b-7efa2059f13f admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:49.296115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a47f13-1af9-412e-889b-7efa2059f13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:49.297413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a47f13-1af9-412e-889b-7efa2059f13f admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:49.298300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a47f13-1af9-412e-889b-7efa2059f13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:49.306361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a47f13-1af9-412e-889b-7efa2059f13f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:49.342707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 858/1700] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:49.343344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb551f3b-473d-437f-be17-21569ac7227c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:49.368208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb551f3b-473d-437f-be17-21569ac7227c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:49.389767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf09e3fd-515d-4c18-9f27-965593e60ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:49.396703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09e3fd-515d-4c18-9f27-965593e60ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:49.396703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09e3fd-515d-4c18-9f27-965593e60ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:49.396703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf09e3fd-515d-4c18-9f27-965593e60ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:49.396703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09e3fd-515d-4c18-9f27-965593e60ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:49.396703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf09e3fd-515d-4c18-9f27-965593e60ac6 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:49.437828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 843/1701] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:49.439787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a47f13-1af9-412e-889b-7efa2059f13f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:49.465855 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a47f13-1af9-412e-889b-7efa2059f13f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:49.477912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0c239-2cb9-4e32-9f48-1cff18b1bfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:49.479160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0c239-2cb9-4e32-9f48-1cff18b1bfa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:49.479714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0c239-2cb9-4e32-9f48-1cff18b1bfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:49.480576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e0c239-2cb9-4e32-9f48-1cff18b1bfa8 admin admin] PATH_INFO: `/v3/projects/1378bfa1199549baba2a955a8545d6a0/users/0a9d160d254c4a1a96ca157f304b753e/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:49.503769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0c239-2cb9-4e32-9f48-1cff18b1bfa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1378bfa1199549baba2a955a8545d6a0, user_id=0a9d160d254c4a1a96ca157f304b753e, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:49.507648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e0c239-2cb9-4e32-9f48-1cff18b1bfa8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:49.554311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e83dbb-28fa-440d-bf1e-8138c4ee3de7 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:49.554717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e83dbb-28fa-440d-bf1e-8138c4ee3de7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:49.555065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e83dbb-28fa-440d-bf1e-8138c4ee3de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:49.559843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 859/1702] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:49 2022] PUT /identity/v3/projects/1378bfa1199549baba2a955a8545d6a0/users/0a9d160d254c4a1a96ca157f304b753e/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:49.642824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-85e83dbb-28fa-440d-bf1e-8138c4ee3de7 None None] MFA Rules not processed for user `0a9d160d254c4a1a96ca157f304b753e`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:49.644139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-85e83dbb-28fa-440d-bf1e-8138c4ee3de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:49.827482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 844/1703] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:49.833159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0c239-2cb9-4e32-9f48-1cff18b1bfa8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:50.041139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:50.107902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e0c239-2cb9-4e32-9f48-1cff18b1bfa8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:50.126818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57e0dd-66de-453f-9885-5ae20d2c1c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:50.128530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57e0dd-66de-453f-9885-5ae20d2c1c27 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:50.133034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57e0dd-66de-453f-9885-5ae20d2c1c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:50.134138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57e0dd-66de-453f-9885-5ae20d2c1c27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:50.135153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57e0dd-66de-453f-9885-5ae20d2c1c27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:50.141501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae57e0dd-66de-453f-9885-5ae20d2c1c27 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:50.174702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92e24f-330e-416a-8292-753b11b9a96c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:50.175426 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92e24f-330e-416a-8292-753b11b9a96c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:50.176054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92e24f-330e-416a-8292-753b11b9a96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:50.240508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-ba92e24f-330e-416a-8292-753b11b9a96c None None] MFA Rules not processed for user `193a35a91a47410f8b12cb2e84b33e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:50.242945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ba92e24f-330e-416a-8292-753b11b9a96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:50.277691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 860/1704] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:49 2022] POST /identity/v3/projects => generated 375 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:50.283334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57e0dd-66de-453f-9885-5ae20d2c1c27 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:50.423263 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 845/1705] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:26:50.424075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92e24f-330e-416a-8292-753b11b9a96c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:50.433442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ba92e24f-330e-416a-8292-753b11b9a96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:50.469750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92e24f-330e-416a-8292-753b11b9a96c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:50.527384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57e0dd-66de-453f-9885-5ae20d2c1c27 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:50.543183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd7c1e-9582-44f9-85b8-16336649aef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:50.543888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd7c1e-9582-44f9-85b8-16336649aef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:50.544174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd7c1e-9582-44f9-85b8-16336649aef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:50.544668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd7c1e-9582-44f9-85b8-16336649aef3 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:50.545140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd7c1e-9582-44f9-85b8-16336649aef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:50.547333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd7c1e-9582-44f9-85b8-16336649aef3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:50.566255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:50.594113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 861/1706] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:50 2022] POST /identity/v3/users => generated 294 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:50.594113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd7c1e-9582-44f9-85b8-16336649aef3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:50.595403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9edd7c1e-9582-44f9-85b8-16336649aef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:50.600754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:50.601658 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:50.602004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:50.602406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] PATH_INFO: `/v3/projects/f6238785c8c04ae1841ee859cb3ebbc8` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:50.606093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6238785c8c04ae1841ee859cb3ebbc8)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:50.608635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:50.750500 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6238785c8c04ae1841ee859cb3ebbc8', 'request_id': 'req-768325a4-3018-4ffa-9f00-851353e531de'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:50.762140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6238785c8c04ae1841ee859cb3ebbc8', 'request_id': 'req-768325a4-3018-4ffa-9f00-851353e531de'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:50.781412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6238785c8c04ae1841ee859cb3ebbc8', 'request_id': 'req-768325a4-3018-4ffa-9f00-851353e531de'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:50.816893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] The token cache is being invalidate because project f6238785c8c04ae1841ee859cb3ebbc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:26:50.817438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:50.824663 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:50.827736 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 846/1707] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:26:50 2022] DELETE /identity/v3/projects/f6238785c8c04ae1841ee859cb3ebbc8 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:50.834125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:50.844331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:50.859839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd7c1e-9582-44f9-85b8-16336649aef3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:51.003727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768325a4-3018-4ffa-9f00-851353e531de admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:51.010410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097d2d2-1862-4d66-8549-c10c2a3e837b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:51.011309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097d2d2-1862-4d66-8549-c10c2a3e837b admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:51.011701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097d2d2-1862-4d66-8549-c10c2a3e837b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:51.012087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097d2d2-1862-4d66-8549-c10c2a3e837b admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:51.013039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097d2d2-1862-4d66-8549-c10c2a3e837b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:51.014916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097d2d2-1862-4d66-8549-c10c2a3e837b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:51.032477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 847/1708] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:51.034157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd74e5-2486-472f-b92f-8c974fcb26b1 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:51.034561 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd74e5-2486-472f-b92f-8c974fcb26b1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:51.035058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd74e5-2486-472f-b92f-8c974fcb26b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:51.042357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:51.061768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2bcd74e5-2486-472f-b92f-8c974fcb26b1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:51.062863 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2bcd74e5-2486-472f-b92f-8c974fcb26b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:51.075292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca098af0-1b01-4f0b-a555-9781d44676a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:51.076226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 848/1709] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:51.077200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd74e5-2486-472f-b92f-8c974fcb26b1 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:51.082989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca098af0-1b01-4f0b-a555-9781d44676a3 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:51.085676 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca098af0-1b01-4f0b-a555-9781d44676a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:51.085676 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca098af0-1b01-4f0b-a555-9781d44676a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:51.090755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd74e5-2486-472f-b92f-8c974fcb26b1 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:51.095853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ca098af0-1b01-4f0b-a555-9781d44676a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:51.102958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae8217-0c4b-461d-b955-0451e3f7f20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:51.104024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae8217-0c4b-461d-b955-0451e3f7f20f admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:51.104414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae8217-0c4b-461d-b955-0451e3f7f20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:51.104780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae8217-0c4b-461d-b955-0451e3f7f20f admin admin] PATH_INFO: `/v3/projects/bf9eed1ff39e42869fc520564c6260c5/users/382b51af0b034254b58882ca15796025/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:51.135571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae8217-0c4b-461d-b955-0451e3f7f20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9eed1ff39e42869fc520564c6260c5, user_id=382b51af0b034254b58882ca15796025, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:51.139700 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae8217-0c4b-461d-b955-0451e3f7f20f admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:51.169000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 849/1710] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:51 2022] PUT /identity/v3/projects/bf9eed1ff39e42869fc520564c6260c5/users/382b51af0b034254b58882ca15796025/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:51.173203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9b0e1-29b0-42d0-9b4e-2c529644a3f3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:51.174091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9b0e1-29b0-42d0-9b4e-2c529644a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:51.174091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f9b0e1-29b0-42d0-9b4e-2c529644a3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:51.230916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e6f9b0e1-29b0-42d0-9b4e-2c529644a3f3 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:51.236590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e6f9b0e1-29b0-42d0-9b4e-2c529644a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:51.260081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca098af0-1b01-4f0b-a555-9781d44676a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:51.261739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca098af0-1b01-4f0b-a555-9781d44676a3 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:51.322457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 862/1711] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 730 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:51.323861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca098af0-1b01-4f0b-a555-9781d44676a3 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:51.330613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:51.355971 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca098af0-1b01-4f0b-a555-9781d44676a3 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:51.409818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 850/1712] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:51.411895 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9b0e1-29b0-42d0-9b4e-2c529644a3f3 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:51.469440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f9b0e1-29b0-42d0-9b4e-2c529644a3f3 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:51.493620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571c8d1-8868-4ade-9f69-260fb69fefb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:51.502019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571c8d1-8868-4ade-9f69-260fb69fefb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:51.503269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571c8d1-8868-4ade-9f69-260fb69fefb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:51.511683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571c8d1-8868-4ade-9f69-260fb69fefb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:51.514346 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571c8d1-8868-4ade-9f69-260fb69fefb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:51.514699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571c8d1-8868-4ade-9f69-260fb69fefb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:51.516408 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9571c8d1-8868-4ade-9f69-260fb69fefb8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:51.596507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 851/1713] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:26:51.597392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571c8d1-8868-4ade-9f69-260fb69fefb8 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:51.606986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9571c8d1-8868-4ade-9f69-260fb69fefb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:51.612583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeab0fe-237c-4765-bc07-10f5576b236e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:51.613599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeab0fe-237c-4765-bc07-10f5576b236e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:51.613890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeab0fe-237c-4765-bc07-10f5576b236e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:51.615571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeab0fe-237c-4765-bc07-10f5576b236e admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:51.616544 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeab0fe-237c-4765-bc07-10f5576b236e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:51.621846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeab0fe-237c-4765-bc07-10f5576b236e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:51.653469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 863/1714] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:51.654651 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeab0fe-237c-4765-bc07-10f5576b236e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:51.677689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeab0fe-237c-4765-bc07-10f5576b236e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:51.684112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571c8d1-8868-4ade-9f69-260fb69fefb8 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:51.696523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:51.698785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:51.699202 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:51.701464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] PATH_INFO: `/v3/projects/3aa3fbd5dc7644079c9a3a5419b52d68` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:51.701464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3aa3fbd5dc7644079c9a3a5419b52d68)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:51.707654 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5028b20-53c1-4b7e-bb54-de46fabf599e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:51.709005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5028b20-53c1-4b7e-bb54-de46fabf599e admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:51.713787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5028b20-53c1-4b7e-bb54-de46fabf599e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:51.718469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5028b20-53c1-4b7e-bb54-de46fabf599e admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:51.721162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:51.726180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5028b20-53c1-4b7e-bb54-de46fabf599e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:51.733822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5028b20-53c1-4b7e-bb54-de46fabf599e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:51.843097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 864/1715] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:51 2022] POST /identity/v3/projects => generated 391 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:51.851996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5028b20-53c1-4b7e-bb54-de46fabf599e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:51.953437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3aa3fbd5dc7644079c9a3a5419b52d68', 'request_id': 'req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:51.982804 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3aa3fbd5dc7644079c9a3a5419b52d68', 'request_id': 'req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:51.996820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3aa3fbd5dc7644079c9a3a5419b52d68', 'request_id': 'req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:52.015016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] The token cache is being invalidate because project 3aa3fbd5dc7644079c9a3a5419b52d68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:26:52.015367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:52.016463 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:52.018380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 852/1716] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:26:51 2022] DELETE /identity/v3/projects/3aa3fbd5dc7644079c9a3a5419b52d68 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:52.019644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:52.021077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:52.026596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5028b20-53c1-4b7e-bb54-de46fabf599e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:52.113044 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:52.292868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ae0f8-13a4-48b3-ac00-533e763a038a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:52.296800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ae0f8-13a4-48b3-ac00-533e763a038a admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:52.297128 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ae0f8-13a4-48b3-ac00-533e763a038a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:52.297361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ae0f8-13a4-48b3-ac00-533e763a038a admin admin] PATH_INFO: `/v3/projects/bf9eed1ff39e42869fc520564c6260c5/users/382b51af0b034254b58882ca15796025/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:52.311745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ae0f8-13a4-48b3-ac00-533e763a038a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf9eed1ff39e42869fc520564c6260c5, user_id=382b51af0b034254b58882ca15796025, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:52.315673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ae0f8-13a4-48b3-ac00-533e763a038a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:52.324635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1090a93-a2bd-45d2-85ca-a716dbc0cc9a admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:52.341115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 865/1717] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:51 2022] PUT /identity/v3/projects/bf9eed1ff39e42869fc520564c6260c5/users/382b51af0b034254b58882ca15796025/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:52.342502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb84b98-0725-456b-b6ce-5799ca16c5d1 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:52.354552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb84b98-0725-456b-b6ce-5799ca16c5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:52.354962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb84b98-0725-456b-b6ce-5799ca16c5d1 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:52.356682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 866/1718] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:26:52 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:26:52.357471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb84b98-0725-456b-b6ce-5799ca16c5d1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:52.361522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ceb84b98-0725-456b-b6ce-5799ca16c5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:52.503877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:52.524262 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb84b98-0725-456b-b6ce-5799ca16c5d1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:52.528424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ebd3f4-754e-42bc-a289-92d73466ff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:52.529131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebd3f4-754e-42bc-a289-92d73466ff5e admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:52.531000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebd3f4-754e-42bc-a289-92d73466ff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:52.531414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ebd3f4-754e-42bc-a289-92d73466ff5e admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:52.532112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebd3f4-754e-42bc-a289-92d73466ff5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:52.538205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e1cf33-f1d9-458a-83fc-ab744257c88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:52.539364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1cf33-f1d9-458a-83fc-ab744257c88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:52.539539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ebd3f4-754e-42bc-a289-92d73466ff5e admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:52.541409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1cf33-f1d9-458a-83fc-ab744257c88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:52.541768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1cf33-f1d9-458a-83fc-ab744257c88b admin admin] PATH_INFO: `/v3/users/757b0aa463f848e6aaacb1d958dc34c0` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:52.545057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1cf33-f1d9-458a-83fc-ab744257c88b admin admin] RBAC: Authorizing `identity:delete_user(user_id=757b0aa463f848e6aaacb1d958dc34c0)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:52.550997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e1cf33-f1d9-458a-83fc-ab744257c88b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:52.613598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 853/1719] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:52 2022] POST /identity/v3/users => generated 296 bytes in 595 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:52.621986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5a333-a23e-4b2e-a68a-f1dccb1e2031 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:52.622305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5a333-a23e-4b2e-a68a-f1dccb1e2031 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:52.622575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5a333-a23e-4b2e-a68a-f1dccb1e2031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:52.645921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-78e1cf33-f1d9-458a-83fc-ab744257c88b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '757b0aa463f848e6aaacb1d958dc34c0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:52.681650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-78e1cf33-f1d9-458a-83fc-ab744257c88b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '757b0aa463f848e6aaacb1d958dc34c0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:52.682480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-78e1cf33-f1d9-458a-83fc-ab744257c88b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '757b0aa463f848e6aaacb1d958dc34c0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:52.692942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-78e1cf33-f1d9-458a-83fc-ab744257c88b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '757b0aa463f848e6aaacb1d958dc34c0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:52.693717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-78e1cf33-f1d9-458a-83fc-ab744257c88b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '757b0aa463f848e6aaacb1d958dc34c0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:52.731406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-fde5a333-a23e-4b2e-a68a-f1dccb1e2031 None None] MFA Rules not processed for user `382b51af0b034254b58882ca15796025`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:52.732744 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-fde5a333-a23e-4b2e-a68a-f1dccb1e2031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:52.745778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 867/1720] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:26:52 2022] DELETE /identity/v3/users/757b0aa463f848e6aaacb1d958dc34c0 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:52.745778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36741380-0e74-4df5-a3b7-ef60978385ab None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:52.745778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36741380-0e74-4df5-a3b7-ef60978385ab None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:52.746240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36741380-0e74-4df5-a3b7-ef60978385ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:52.837661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-36741380-0e74-4df5-a3b7-ef60978385ab None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:52.849006 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-36741380-0e74-4df5-a3b7-ef60978385ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:52.938016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:52.969056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 854/1721] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:52.970156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5a333-a23e-4b2e-a68a-f1dccb1e2031 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:52.975368 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-fde5a333-a23e-4b2e-a68a-f1dccb1e2031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:53.078110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 868/1722] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:26:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:53.079082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36741380-0e74-4df5-a3b7-ef60978385ab None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:53.082476 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-36741380-0e74-4df5-a3b7-ef60978385ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:53.139792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5a333-a23e-4b2e-a68a-f1dccb1e2031 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:53.157423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56988c6d-3b8d-4511-8d8e-0bf490c0918b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:53.158122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56988c6d-3b8d-4511-8d8e-0bf490c0918b admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:53.158478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56988c6d-3b8d-4511-8d8e-0bf490c0918b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:53.158788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56988c6d-3b8d-4511-8d8e-0bf490c0918b admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:53.159438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56988c6d-3b8d-4511-8d8e-0bf490c0918b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:53.161078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56988c6d-3b8d-4511-8d8e-0bf490c0918b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:53.187013 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 855/1723] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:53.195509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56988c6d-3b8d-4511-8d8e-0bf490c0918b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:53.213391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56988c6d-3b8d-4511-8d8e-0bf490c0918b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:53.227936 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4312-2fd9-4792-98ec-69d39d9991fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:53.228645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4312-2fd9-4792-98ec-69d39d9991fd admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:53.228929 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4312-2fd9-4792-98ec-69d39d9991fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:53.229206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4312-2fd9-4792-98ec-69d39d9991fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:53.229952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a4312-2fd9-4792-98ec-69d39d9991fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:53.231637 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a4312-2fd9-4792-98ec-69d39d9991fd admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:53.263758 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 856/1724] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:53.264905 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4312-2fd9-4792-98ec-69d39d9991fd admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:53.267788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36741380-0e74-4df5-a3b7-ef60978385ab None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:53.281308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c87d5-30a3-4e1a-8d2f-6234013c29ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:53.282151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c87d5-30a3-4e1a-8d2f-6234013c29ee service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:53.282437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c87d5-30a3-4e1a-8d2f-6234013c29ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:53.282746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c87d5-30a3-4e1a-8d2f-6234013c29ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:53.284356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4312-2fd9-4792-98ec-69d39d9991fd admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:53.290875 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d58c87d5-30a3-4e1a-8d2f-6234013c29ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:53.306687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486cf64-63fd-490c-903c-f81ce0e00592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:53.306687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486cf64-63fd-490c-903c-f81ce0e00592 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:53.306687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486cf64-63fd-490c-903c-f81ce0e00592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:53.306687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486cf64-63fd-490c-903c-f81ce0e00592 admin admin] PATH_INFO: `/v3/projects/521fa955dcb84ec695b808ce7d32f913/users/b785d0c964084690990c1b2245b0dd3e/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:53.363225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486cf64-63fd-490c-903c-f81ce0e00592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=521fa955dcb84ec695b808ce7d32f913, user_id=b785d0c964084690990c1b2245b0dd3e, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:53.365945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486cf64-63fd-490c-903c-f81ce0e00592 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:53.427363 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 857/1725] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:53 2022] PUT /identity/v3/projects/521fa955dcb84ec695b808ce7d32f913/users/b785d0c964084690990c1b2245b0dd3e/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:53.440302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7e57b-8b03-4535-9ea2-b6d68c504a84 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:53.440653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7e57b-8b03-4535-9ea2-b6d68c504a84 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:53.440889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7e57b-8b03-4535-9ea2-b6d68c504a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:53.471762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c87d5-30a3-4e1a-8d2f-6234013c29ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:53.473264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c87d5-30a3-4e1a-8d2f-6234013c29ee service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:53.502021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 869/1726] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:53.517902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-22a7e57b-8b03-4535-9ea2-b6d68c504a84 None None] MFA Rules not processed for user `b785d0c964084690990c1b2245b0dd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:53.519134 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-22a7e57b-8b03-4535-9ea2-b6d68c504a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:53.532389 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c87d5-30a3-4e1a-8d2f-6234013c29ee service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:53.533865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d58c87d5-30a3-4e1a-8d2f-6234013c29ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:53.565215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c87d5-30a3-4e1a-8d2f-6234013c29ee service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:53.573341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:53.717720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ff674-16e3-417b-81fd-10687987ef32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:53.718389 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ff674-16e3-417b-81fd-10687987ef32 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:53.718711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ff674-16e3-417b-81fd-10687987ef32 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:53.718967 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ff674-16e3-417b-81fd-10687987ef32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:53.742138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ff674-16e3-417b-81fd-10687987ef32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:53.743927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ff674-16e3-417b-81fd-10687987ef32 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:53.752311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 858/1727] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:53.752311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7e57b-8b03-4535-9ea2-b6d68c504a84 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:53.802118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 870/1728] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:53.924055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7e57b-8b03-4535-9ea2-b6d68c504a84 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:53.931342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c865c5b-acdc-4259-b51f-5c88ba2fad0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:53.932161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c865c5b-acdc-4259-b51f-5c88ba2fad0c admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:53.932462 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c865c5b-acdc-4259-b51f-5c88ba2fad0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:53.933326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c865c5b-acdc-4259-b51f-5c88ba2fad0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:53.934370 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c865c5b-acdc-4259-b51f-5c88ba2fad0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:53.937174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c865c5b-acdc-4259-b51f-5c88ba2fad0c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:53.992898 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 859/1729] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:53 2022] POST /identity/v3/projects => generated 391 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:53.995974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ff674-16e3-417b-81fd-10687987ef32 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:54.043974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ff674-16e3-417b-81fd-10687987ef32 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:54.070409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:54.231705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806099b2-9778-47ea-9906-3b826e36356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:54.232568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806099b2-9778-47ea-9906-3b826e36356d admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:54.232856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806099b2-9778-47ea-9906-3b826e36356d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:54.233132 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806099b2-9778-47ea-9906-3b826e36356d admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:54.233830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806099b2-9778-47ea-9906-3b826e36356d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:54.236500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806099b2-9778-47ea-9906-3b826e36356d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:54.314780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 871/1730] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:53 2022] POST /identity/v3/users => generated 302 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:54.318621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c865c5b-acdc-4259-b51f-5c88ba2fad0c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:54.331853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c865c5b-acdc-4259-b51f-5c88ba2fad0c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:54.368315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c63681-55f7-4296-834d-2e41713cad59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:54.369178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c63681-55f7-4296-834d-2e41713cad59 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:54.369568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c63681-55f7-4296-834d-2e41713cad59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:54.370122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c63681-55f7-4296-834d-2e41713cad59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:54.372250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c63681-55f7-4296-834d-2e41713cad59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:54.380210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c63681-55f7-4296-834d-2e41713cad59 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:54.412102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 860/1731] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:26:54.413592 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806099b2-9778-47ea-9906-3b826e36356d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:54.436642 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806099b2-9778-47ea-9906-3b826e36356d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:54.481537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6520a-ad2a-44a6-9d5f-0f1db72229c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:54.484367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6520a-ad2a-44a6-9d5f-0f1db72229c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:54.484367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6520a-ad2a-44a6-9d5f-0f1db72229c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:54.484367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6520a-ad2a-44a6-9d5f-0f1db72229c0 admin admin] PATH_INFO: `/v3/projects/4380b118062f4d2aac2fba5423a60807/users/a880ded9be7d442cb05f1848a454ed13/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:54.547321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6520a-ad2a-44a6-9d5f-0f1db72229c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4380b118062f4d2aac2fba5423a60807, user_id=a880ded9be7d442cb05f1848a454ed13, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:54.556122 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6520a-ad2a-44a6-9d5f-0f1db72229c0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:54.629659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 872/1732] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:54 2022] PUT /identity/v3/projects/4380b118062f4d2aac2fba5423a60807/users/a880ded9be7d442cb05f1848a454ed13/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:26:54.633413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c63681-55f7-4296-834d-2e41713cad59 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:54.652127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c63681-55f7-4296-834d-2e41713cad59 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:54.785212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330aa6e4-bcfd-4a46-a463-a47979459be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:54.786077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330aa6e4-bcfd-4a46-a463-a47979459be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:54.786459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330aa6e4-bcfd-4a46-a463-a47979459be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:54.786858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330aa6e4-bcfd-4a46-a463-a47979459be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:54.787700 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330aa6e4-bcfd-4a46-a463-a47979459be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:54.789424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330aa6e4-bcfd-4a46-a463-a47979459be0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:54.815943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 861/1733] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:26:54.820180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6520a-ad2a-44a6-9d5f-0f1db72229c0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:54.832833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6520a-ad2a-44a6-9d5f-0f1db72229c0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:54.852914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf782f-0f59-4930-af28-a1e4a12e3744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:54.853953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf782f-0f59-4930-af28-a1e4a12e3744 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:54.854345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf782f-0f59-4930-af28-a1e4a12e3744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:54.854887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf782f-0f59-4930-af28-a1e4a12e3744 admin admin] PATH_INFO: `/v3/projects/4380b118062f4d2aac2fba5423a60807/users/a880ded9be7d442cb05f1848a454ed13/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:54.869853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf782f-0f59-4930-af28-a1e4a12e3744 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4380b118062f4d2aac2fba5423a60807, user_id=a880ded9be7d442cb05f1848a454ed13, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:54.873058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf782f-0f59-4930-af28-a1e4a12e3744 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:54.911146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 873/1734] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:54 2022] PUT /identity/v3/projects/4380b118062f4d2aac2fba5423a60807/users/a880ded9be7d442cb05f1848a454ed13/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:54.922277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575af86b-fbd3-46f9-b7bf-3ed53dfcc002 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:54.922728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575af86b-fbd3-46f9-b7bf-3ed53dfcc002 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:54.923113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575af86b-fbd3-46f9-b7bf-3ed53dfcc002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:54.977006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-575af86b-fbd3-46f9-b7bf-3ed53dfcc002 None None] MFA Rules not processed for user `a880ded9be7d442cb05f1848a454ed13`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:54.979205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-575af86b-fbd3-46f9-b7bf-3ed53dfcc002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:55.032042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:55.161841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 862/1735] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:55.178114 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf782f-0f59-4930-af28-a1e4a12e3744 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:55.334951 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf782f-0f59-4930-af28-a1e4a12e3744 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:55.351300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201483e-9941-436c-bfe1-c594b8b0c2d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:55.352207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201483e-9941-436c-bfe1-c594b8b0c2d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:55.352571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201483e-9941-436c-bfe1-c594b8b0c2d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:55.353012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201483e-9941-436c-bfe1-c594b8b0c2d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:55.380106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201483e-9941-436c-bfe1-c594b8b0c2d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:55.392826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201483e-9941-436c-bfe1-c594b8b0c2d6 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:55.414065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 874/1736] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:55.585341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575af86b-fbd3-46f9-b7bf-3ed53dfcc002 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:55.587545 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-575af86b-fbd3-46f9-b7bf-3ed53dfcc002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:55.628686 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575af86b-fbd3-46f9-b7bf-3ed53dfcc002 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:55.639903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee804e59-88f6-4282-9f96-8729443d82cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:55.640591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee804e59-88f6-4282-9f96-8729443d82cf service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:55.640876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee804e59-88f6-4282-9f96-8729443d82cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:55.641155 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee804e59-88f6-4282-9f96-8729443d82cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:55.788870 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee804e59-88f6-4282-9f96-8729443d82cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:55.795785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee804e59-88f6-4282-9f96-8729443d82cf service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:55.816508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 863/1737] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:26:56.429694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201483e-9941-436c-bfe1-c594b8b0c2d6 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:56.433034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a201483e-9941-436c-bfe1-c594b8b0c2d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:56.642485 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201483e-9941-436c-bfe1-c594b8b0c2d6 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:56.655008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:56.656174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:56.656669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:56.657144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] PATH_INFO: `/v3/projects/26306aaa359c45ceaad844030abc3403` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:56.659274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26306aaa359c45ceaad844030abc3403)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:56.665277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:56.673026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:56.816191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26306aaa359c45ceaad844030abc3403', 'request_id': 'req-8b18c278-c587-49a8-948d-5cc946e99b24'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:56.831884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26306aaa359c45ceaad844030abc3403', 'request_id': 'req-8b18c278-c587-49a8-948d-5cc946e99b24'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:56.881015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26306aaa359c45ceaad844030abc3403', 'request_id': 'req-8b18c278-c587-49a8-948d-5cc946e99b24'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:56.921202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] The token cache is being invalidate because project 26306aaa359c45ceaad844030abc3403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:26:56.923121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:56.925100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8b18c278-c587-49a8-948d-5cc946e99b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:26:56.928459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 875/1738] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:26:56 2022] DELETE /identity/v3/projects/26306aaa359c45ceaad844030abc3403 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:56.933205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e16f4-4827-488d-a0cf-721ef1d6f986 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:56.934132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e16f4-4827-488d-a0cf-721ef1d6f986 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:56.934694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481e16f4-4827-488d-a0cf-721ef1d6f986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:57.003256 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-481e16f4-4827-488d-a0cf-721ef1d6f986 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:57.005267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-481e16f4-4827-488d-a0cf-721ef1d6f986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:57.194472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 864/1739] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:57.202826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc470cc-b31f-41d8-8d40-7fd78a56daf3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:57.203373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc470cc-b31f-41d8-8d40-7fd78a56daf3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:57.204341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc470cc-b31f-41d8-8d40-7fd78a56daf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:57.289387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-fcc470cc-b31f-41d8-8d40-7fd78a56daf3 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:57.300904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fcc470cc-b31f-41d8-8d40-7fd78a56daf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:57.360365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e16f4-4827-488d-a0cf-721ef1d6f986 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:57.368782 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 876/1740] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:57.404869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481e16f4-4827-488d-a0cf-721ef1d6f986 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:57.442675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b46ce-9e5d-43bb-aef8-b9466a09079e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:57.447214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b46ce-9e5d-43bb-aef8-b9466a09079e admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:57.450768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b46ce-9e5d-43bb-aef8-b9466a09079e admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:57.452542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b46ce-9e5d-43bb-aef8-b9466a09079e admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:57.454282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b46ce-9e5d-43bb-aef8-b9466a09079e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:26:57.457924 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b46ce-9e5d-43bb-aef8-b9466a09079e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:57.462342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b46ce-9e5d-43bb-aef8-b9466a09079e admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:57.559336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc470cc-b31f-41d8-8d40-7fd78a56daf3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:57.562199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 865/1741] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:26:57.581442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc470cc-b31f-41d8-8d40-7fd78a56daf3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:57.597381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299982b8-46b1-4d67-9235-d3a56bffcb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:57.598116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299982b8-46b1-4d67-9235-d3a56bffcb4b admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:57.598385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299982b8-46b1-4d67-9235-d3a56bffcb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:57.598695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299982b8-46b1-4d67-9235-d3a56bffcb4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:57.599407 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299982b8-46b1-4d67-9235-d3a56bffcb4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:57.609625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299982b8-46b1-4d67-9235-d3a56bffcb4b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:57.718110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 877/1742] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:57 2022] POST /identity/v3/projects => generated 407 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:57.722314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b46ce-9e5d-43bb-aef8-b9466a09079e admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:57.746581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b46ce-9e5d-43bb-aef8-b9466a09079e admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:57.911955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:57.972151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9483-9914-4ae1-86ff-007009400cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:57.972859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9483-9914-4ae1-86ff-007009400cbf admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:57.973152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9483-9914-4ae1-86ff-007009400cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:57.973422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9483-9914-4ae1-86ff-007009400cbf admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:57.974126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac9483-9914-4ae1-86ff-007009400cbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:57.976147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac9483-9914-4ae1-86ff-007009400cbf admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:58.030853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 866/1743] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:57 2022] POST /identity/v3/users => generated 304 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:58.036896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299982b8-46b1-4d67-9235-d3a56bffcb4b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:58.052196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299982b8-46b1-4d67-9235-d3a56bffcb4b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:58.068577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8eb79-0e44-440c-b473-f87403afae49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:58.072081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8eb79-0e44-440c-b473-f87403afae49 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:58.073414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8eb79-0e44-440c-b473-f87403afae49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:58.074958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8eb79-0e44-440c-b473-f87403afae49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:58.076815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8eb79-0e44-440c-b473-f87403afae49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:58.080962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8eb79-0e44-440c-b473-f87403afae49 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:58.137204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 878/1744] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:58.140532 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9483-9914-4ae1-86ff-007009400cbf admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:58.154918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9483-9914-4ae1-86ff-007009400cbf admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:58.183150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47ffa4-b2c8-4c55-9cce-560ce13431bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:58.183888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47ffa4-b2c8-4c55-9cce-560ce13431bb admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:58.184162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47ffa4-b2c8-4c55-9cce-560ce13431bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:58.184443 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47ffa4-b2c8-4c55-9cce-560ce13431bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:58.185201 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47ffa4-b2c8-4c55-9cce-560ce13431bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:58.186953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47ffa4-b2c8-4c55-9cce-560ce13431bb admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:58.202943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 867/1745] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:58.207251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8eb79-0e44-440c-b473-f87403afae49 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:58.220225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8eb79-0e44-440c-b473-f87403afae49 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:58.235268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bb259-1c41-49fb-a218-c5fc22fc6100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:58.236207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bb259-1c41-49fb-a218-c5fc22fc6100 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:58.236494 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bb259-1c41-49fb-a218-c5fc22fc6100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:58.236771 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25bb259-1c41-49fb-a218-c5fc22fc6100 admin admin] PATH_INFO: `/v3/projects/53e1947047fb4658bc271ae0ddf85cb3/users/0afa6ba21f64474b803463c1773848b9/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:58.271403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bb259-1c41-49fb-a218-c5fc22fc6100 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e1947047fb4658bc271ae0ddf85cb3, user_id=0afa6ba21f64474b803463c1773848b9, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:58.276876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25bb259-1c41-49fb-a218-c5fc22fc6100 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:58.314192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 879/1746] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:58 2022] PUT /identity/v3/projects/53e1947047fb4658bc271ae0ddf85cb3/users/0afa6ba21f64474b803463c1773848b9/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:58.316020 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907dd91-7a7d-448f-9ffc-30a2843cb8eb None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:58.316317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907dd91-7a7d-448f-9ffc-30a2843cb8eb None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:58.316606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907dd91-7a7d-448f-9ffc-30a2843cb8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:58.380586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d907dd91-7a7d-448f-9ffc-30a2843cb8eb None None] MFA Rules not processed for user `0afa6ba21f64474b803463c1773848b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:58.381764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d907dd91-7a7d-448f-9ffc-30a2843cb8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:58.593169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 868/1747] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:58 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:58.597869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bb259-1c41-49fb-a218-c5fc22fc6100 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:58.753078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25bb259-1c41-49fb-a218-c5fc22fc6100 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:58.763828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096193a6-68d7-4c28-be99-c14aa96ea71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:58.764582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096193a6-68d7-4c28-be99-c14aa96ea71d admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:58.764876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096193a6-68d7-4c28-be99-c14aa96ea71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:58.765161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096193a6-68d7-4c28-be99-c14aa96ea71d admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:58.765868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096193a6-68d7-4c28-be99-c14aa96ea71d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:58.767906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096193a6-68d7-4c28-be99-c14aa96ea71d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:58.833455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 880/1748] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:26:58 2022] POST /identity/v3/projects => generated 403 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:58.838347 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907dd91-7a7d-448f-9ffc-30a2843cb8eb None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:58.853494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907dd91-7a7d-448f-9ffc-30a2843cb8eb None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:58.972539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870492eb-1c30-461f-a037-95d6eefc0a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:58.972539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870492eb-1c30-461f-a037-95d6eefc0a01 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:58.972539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870492eb-1c30-461f-a037-95d6eefc0a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:58.972539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870492eb-1c30-461f-a037-95d6eefc0a01 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:58.973467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870492eb-1c30-461f-a037-95d6eefc0a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:58.973467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870492eb-1c30-461f-a037-95d6eefc0a01 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:59.015470 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 869/1749] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:26:58 2022] POST /identity/v3/users => generated 308 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:26:59.019800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096193a6-68d7-4c28-be99-c14aa96ea71d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:59.040217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096193a6-68d7-4c28-be99-c14aa96ea71d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:59.058125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ba81b-6dff-45f3-b7b9-e82fe12a6139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:59.059168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ba81b-6dff-45f3-b7b9-e82fe12a6139 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:59.059659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ba81b-6dff-45f3-b7b9-e82fe12a6139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:59.060135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ba81b-6dff-45f3-b7b9-e82fe12a6139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:59.061901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ba81b-6dff-45f3-b7b9-e82fe12a6139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:59.063867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ba81b-6dff-45f3-b7b9-e82fe12a6139 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:59.080143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 881/1750] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:59.083808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870492eb-1c30-461f-a037-95d6eefc0a01 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:59.099211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870492eb-1c30-461f-a037-95d6eefc0a01 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:59.105397 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c02e70-af1e-4543-b49b-1490a95d4d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:59.107029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c02e70-af1e-4543-b49b-1490a95d4d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:59.107457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c02e70-af1e-4543-b49b-1490a95d4d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:59.107850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c02e70-af1e-4543-b49b-1490a95d4d57 admin admin] PATH_INFO: `/v3/projects/d076ac6ad92e4732886152da39a14f32/users/f8cd031247704e8796ea8b6534e2610d/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:59.137145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c02e70-af1e-4543-b49b-1490a95d4d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d076ac6ad92e4732886152da39a14f32, user_id=f8cd031247704e8796ea8b6534e2610d, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:59.145367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c02e70-af1e-4543-b49b-1490a95d4d57 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:59.185261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 870/1751] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:59 2022] PUT /identity/v3/projects/d076ac6ad92e4732886152da39a14f32/users/f8cd031247704e8796ea8b6534e2610d/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:59.189136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ba81b-6dff-45f3-b7b9-e82fe12a6139 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:59.195139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:26:59.207129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ba81b-6dff-45f3-b7b9-e82fe12a6139 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:59.346722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48d22b-ebdc-4040-b466-5c1d76f891ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:59.348175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48d22b-ebdc-4040-b466-5c1d76f891ab admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:59.348469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48d22b-ebdc-4040-b466-5c1d76f891ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:59.348749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48d22b-ebdc-4040-b466-5c1d76f891ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:59.349499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48d22b-ebdc-4040-b466-5c1d76f891ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:59.351711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48d22b-ebdc-4040-b466-5c1d76f891ab admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:59.375725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 882/1752] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:26:59.379896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c02e70-af1e-4543-b49b-1490a95d4d57 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:59.401697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c02e70-af1e-4543-b49b-1490a95d4d57 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:26:59.426680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376b176-781f-4cc1-bcaf-8992094ee486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:26:59.427608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376b176-781f-4cc1-bcaf-8992094ee486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:59.427973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376b176-781f-4cc1-bcaf-8992094ee486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:59.428648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376b176-781f-4cc1-bcaf-8992094ee486 admin admin] PATH_INFO: `/v3/projects/d076ac6ad92e4732886152da39a14f32/users/f8cd031247704e8796ea8b6534e2610d/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:59.434700 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376b176-781f-4cc1-bcaf-8992094ee486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d076ac6ad92e4732886152da39a14f32, user_id=f8cd031247704e8796ea8b6534e2610d, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:26:59.437267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376b176-781f-4cc1-bcaf-8992094ee486 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:26:59.493820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 871/1753] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:26:59 2022] PUT /identity/v3/projects/d076ac6ad92e4732886152da39a14f32/users/f8cd031247704e8796ea8b6534e2610d/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:26:59.505716 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac1f16-9006-4978-a856-08d485f4e2be None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:26:59.506260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac1f16-9006-4978-a856-08d485f4e2be None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:26:59.506751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac1f16-9006-4978-a856-08d485f4e2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:26:59.616895 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-97ac1f16-9006-4978-a856-08d485f4e2be None None] MFA Rules not processed for user `f8cd031247704e8796ea8b6534e2610d`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:26:59.624028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-97ac1f16-9006-4978-a856-08d485f4e2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:59.829876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 883/1754] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:26:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:26:59.840082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376b176-781f-4cc1-bcaf-8992094ee486 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:26:59.843313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9376b176-781f-4cc1-bcaf-8992094ee486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:26:59.850466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:00.065645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376b176-781f-4cc1-bcaf-8992094ee486 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:00.080504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0374e33-a3d7-4058-8d6b-d15ce5adc7cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:00.081333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0374e33-a3d7-4058-8d6b-d15ce5adc7cf service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:00.081462 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0374e33-a3d7-4058-8d6b-d15ce5adc7cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:00.081819 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0374e33-a3d7-4058-8d6b-d15ce5adc7cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:00.109259 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0374e33-a3d7-4058-8d6b-d15ce5adc7cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:00.110987 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0374e33-a3d7-4058-8d6b-d15ce5adc7cf service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:00.153005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 872/1755] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:26:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:00.276851 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75abae-7d6c-4db6-b660-f2b07cb3073e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:00.277319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75abae-7d6c-4db6-b660-f2b07cb3073e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:00.277882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75abae-7d6c-4db6-b660-f2b07cb3073e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:00.311926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-cf75abae-7d6c-4db6-b660-f2b07cb3073e None None] MFA Rules not processed for user `193a35a91a47410f8b12cb2e84b33e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:00.313155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cf75abae-7d6c-4db6-b660-f2b07cb3073e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:00.443652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0374e33-a3d7-4058-8d6b-d15ce5adc7cf service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:00.455085 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0374e33-a3d7-4058-8d6b-d15ce5adc7cf service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:00.462577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837976e9-50ce-4797-9523-d0fefff7fcd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:00.463247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837976e9-50ce-4797-9523-d0fefff7fcd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:00.463531 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837976e9-50ce-4797-9523-d0fefff7fcd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:00.463806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837976e9-50ce-4797-9523-d0fefff7fcd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:00.469976 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 884/1756] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:00.622831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837976e9-50ce-4797-9523-d0fefff7fcd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:00.625079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837976e9-50ce-4797-9523-d0fefff7fcd6 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:00.653209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 873/1757] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:00 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:00.964576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed164a-c108-416a-93ff-d7834b8ed899 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:00.964996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed164a-c108-416a-93ff-d7834b8ed899 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:00.965141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed164a-c108-416a-93ff-d7834b8ed899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:01.040040 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c25e8-12d7-467c-b93d-9aad10857ccf None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:01.040351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c25e8-12d7-467c-b93d-9aad10857ccf None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:01.040617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c25e8-12d7-467c-b93d-9aad10857ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:01.051034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1aed164a-c108-416a-93ff-d7834b8ed899 None None] MFA Rules not processed for user `127a6938bc5c4416a9a502d9064344cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:01.052148 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1aed164a-c108-416a-93ff-d7834b8ed899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:01.087528 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-3a1c25e8-12d7-467c-b93d-9aad10857ccf None None] MFA Rules not processed for user `8632496bdc1b4d06a74caf483c8b0a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:01.091784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3a1c25e8-12d7-467c-b93d-9aad10857ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:01.101534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:01.231843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 874/1758] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:01.250577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 885/1759] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:01.255627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1929f8-16cd-4858-ad6d-17cb0aaef543 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:01.256790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1929f8-16cd-4858-ad6d-17cb0aaef543 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:01.257303 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1929f8-16cd-4858-ad6d-17cb0aaef543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:01.304188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-9a1929f8-16cd-4858-ad6d-17cb0aaef543 None None] MFA Rules not processed for user `127a6938bc5c4416a9a502d9064344cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:01.307492 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9a1929f8-16cd-4858-ad6d-17cb0aaef543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:01.323566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 875/1760] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:02.022249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed164a-c108-416a-93ff-d7834b8ed899 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:02.039890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed164a-c108-416a-93ff-d7834b8ed899 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:02.051237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237c41-8411-4b9f-9e6d-bda195165f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:02.052334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237c41-8411-4b9f-9e6d-bda195165f3e service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:02.052829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237c41-8411-4b9f-9e6d-bda195165f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:02.054292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45237c41-8411-4b9f-9e6d-bda195165f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:02.059376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-45237c41-8411-4b9f-9e6d-bda195165f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:02.078595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a7c5c-0c49-4d2f-908b-f8b07f49488d None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:02.078886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a7c5c-0c49-4d2f-908b-f8b07f49488d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:02.079954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a7c5c-0c49-4d2f-908b-f8b07f49488d None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:02.081658 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 876/1761] 10.222.0.93 () {54 vars in 912 bytes} [Fri Jun 3 15:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:27:02.091948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71e662-a355-415a-a7f2-ee32ad49627e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:02.093702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71e662-a355-415a-a7f2-ee32ad49627e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:02.093991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71e662-a355-415a-a7f2-ee32ad49627e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:02.129721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-bb71e662-a355-415a-a7f2-ee32ad49627e None None] MFA Rules not processed for user `e3f887d5bbcd424884537b95affd96f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:02.146119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-bb71e662-a355-415a-a7f2-ee32ad49627e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:02.245738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:02.251506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45237c41-8411-4b9f-9e6d-bda195165f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:02.253273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45237c41-8411-4b9f-9e6d-bda195165f3e service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:02.278891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 886/1762] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:02.280168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237c41-8411-4b9f-9e6d-bda195165f3e service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:02.284874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 877/1763] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 03 15:27:02.285415 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-45237c41-8411-4b9f-9e6d-bda195165f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:02.432350 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71e662-a355-415a-a7f2-ee32ad49627e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:02.434297 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-bb71e662-a355-415a-a7f2-ee32ad49627e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:02.462178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71e662-a355-415a-a7f2-ee32ad49627e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:02.488935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda05f5-e6d3-442e-a567-8123ed967c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:02.492818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda05f5-e6d3-442e-a567-8123ed967c1b service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:02.497723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda05f5-e6d3-442e-a567-8123ed967c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:02.497723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda05f5-e6d3-442e-a567-8123ed967c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:02.534728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda05f5-e6d3-442e-a567-8123ed967c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:02.542673 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda05f5-e6d3-442e-a567-8123ed967c1b service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:02.579816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45237c41-8411-4b9f-9e6d-bda195165f3e service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:02.580906 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 878/1764] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:02.596286 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:02.597698 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:02.598194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:02.601481 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] PATH_INFO: `/v3/users/b785d0c964084690990c1b2245b0dd3e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:02.607364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b785d0c964084690990c1b2245b0dd3e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:02.612815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:02.843446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b785d0c964084690990c1b2245b0dd3e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:02.879969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:02.915558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b785d0c964084690990c1b2245b0dd3e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:02.922022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b785d0c964084690990c1b2245b0dd3e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:02.953647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b785d0c964084690990c1b2245b0dd3e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:02.961043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b785d0c964084690990c1b2245b0dd3e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:03.012483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 887/1765] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:02 2022] DELETE /identity/v3/users/b785d0c964084690990c1b2245b0dd3e => generated 0 bytes in 733 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:03.017222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb0002-4b22-4881-b56c-dab7f861b76a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:03.018018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb0002-4b22-4881-b56c-dab7f861b76a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:03.018328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb0002-4b22-4881-b56c-dab7f861b76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:03.074565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-cceb0002-4b22-4881-b56c-dab7f861b76a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:03.075843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-cceb0002-4b22-4881-b56c-dab7f861b76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:03.271376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 879/1766] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:03.312240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:03.314508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:03.486989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb0002-4b22-4881-b56c-dab7f861b76a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:03.488571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-cceb0002-4b22-4881-b56c-dab7f861b76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:03.504303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ec959-1e45-44e9-b7a7-fff8fe163c63 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:03.511407 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2d5f3-4803-4ad0-9173-47547c9ae48f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:03.512087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2d5f3-4803-4ad0-9173-47547c9ae48f service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:03.514781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2d5f3-4803-4ad0-9173-47547c9ae48f service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:03.515104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2d5f3-4803-4ad0-9173-47547c9ae48f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:03.545209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2d5f3-4803-4ad0-9173-47547c9ae48f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:03.548022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2d5f3-4803-4ad0-9173-47547c9ae48f service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:03.585237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 888/1767] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:03.782503 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb0002-4b22-4881-b56c-dab7f861b76a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:03.807953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d059291-908d-4cf5-bf9e-2ed7f7e7d09b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:03.808551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d059291-908d-4cf5-bf9e-2ed7f7e7d09b service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:03.808843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d059291-908d-4cf5-bf9e-2ed7f7e7d09b service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:03.809126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d059291-908d-4cf5-bf9e-2ed7f7e7d09b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:03.813388 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2d059291-908d-4cf5-bf9e-2ed7f7e7d09b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:03.963497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2ee63-84e9-4e0b-8e43-f0a9cb03b994 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:03.963930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2ee63-84e9-4e0b-8e43-f0a9cb03b994 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:03.964844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2ee63-84e9-4e0b-8e43-f0a9cb03b994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:03.989611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d059291-908d-4cf5-bf9e-2ed7f7e7d09b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:03.992216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d059291-908d-4cf5-bf9e-2ed7f7e7d09b service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:04.020652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 880/1768] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:04.100783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ede2ee63-84e9-4e0b-8e43-f0a9cb03b994 None None] MFA Rules not processed for user `47d7587d10b3487688ed251a85b83c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:04.107103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ede2ee63-84e9-4e0b-8e43-f0a9cb03b994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:04.348932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 889/1769] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:04.484089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d059291-908d-4cf5-bf9e-2ed7f7e7d09b service nova] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:04.487867 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2d059291-908d-4cf5-bf9e-2ed7f7e7d09b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:04.561234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d059291-908d-4cf5-bf9e-2ed7f7e7d09b service nova] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:04.581716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:04.586932 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:04.589211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:04.591419 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] PATH_INFO: `/v3/projects/521fa955dcb84ec695b808ce7d32f913` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:04.600253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=521fa955dcb84ec695b808ce7d32f913)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:04.626776 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:04.630000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2ee63-84e9-4e0b-8e43-f0a9cb03b994 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:04.631841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ede2ee63-84e9-4e0b-8e43-f0a9cb03b994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:04.651388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2ee63-84e9-4e0b-8e43-f0a9cb03b994 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:04.658983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad02fde-392b-408a-96bb-a15af79cd88d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:04.660302 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad02fde-392b-408a-96bb-a15af79cd88d service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:04.660769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad02fde-392b-408a-96bb-a15af79cd88d service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:04.661221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad02fde-392b-408a-96bb-a15af79cd88d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:04.673665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad02fde-392b-408a-96bb-a15af79cd88d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:04.676860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad02fde-392b-408a-96bb-a15af79cd88d service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:04.679265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:04.701917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 890/1770] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:04.825588 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '521fa955dcb84ec695b808ce7d32f913', 'request_id': 'req-cc239a89-c926-4514-8a83-1ac5761546d7'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:04.863862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '521fa955dcb84ec695b808ce7d32f913', 'request_id': 'req-cc239a89-c926-4514-8a83-1ac5761546d7'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:04.888519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '521fa955dcb84ec695b808ce7d32f913', 'request_id': 'req-cc239a89-c926-4514-8a83-1ac5761546d7'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:04.920787 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] The token cache is being invalidate because project 521fa955dcb84ec695b808ce7d32f913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:04.921383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:04.922192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cc239a89-c926-4514-8a83-1ac5761546d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:04.924294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 881/1771] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:04 2022] DELETE /identity/v3/projects/521fa955dcb84ec695b808ce7d32f913 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:04.929281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad02fde-392b-408a-96bb-a15af79cd88d service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:04.948994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad02fde-392b-408a-96bb-a15af79cd88d service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:05.138808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:05.177162 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:05.178783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:05.179351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:05.180552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] PATH_INFO: `/v3/users/a880ded9be7d442cb05f1848a454ed13` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:05.185472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a880ded9be7d442cb05f1848a454ed13)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:05.188922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:05.378892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a880ded9be7d442cb05f1848a454ed13'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:05.431541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a880ded9be7d442cb05f1848a454ed13'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:05.433237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a880ded9be7d442cb05f1848a454ed13'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:05.463824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a880ded9be7d442cb05f1848a454ed13'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:05.468087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a880ded9be7d442cb05f1848a454ed13'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:05.499365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 891/1772] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:04 2022] DELETE /identity/v3/users/a880ded9be7d442cb05f1848a454ed13 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:05.970307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cb096-ba1b-4d1a-8f84-00b3ec671895 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:05.971490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cb096-ba1b-4d1a-8f84-00b3ec671895 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:05.971928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cb096-ba1b-4d1a-8f84-00b3ec671895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:06.011930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-f29cb096-ba1b-4d1a-8f84-00b3ec671895 None None] MFA Rules not processed for user `127a6938bc5c4416a9a502d9064344cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:06.014834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f29cb096-ba1b-4d1a-8f84-00b3ec671895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:06.219491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 882/1773] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:06.457483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:06.465806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:06.664974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae6673-6989-485a-8fe4-838b513184f2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:06.681487 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:06.682397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:06.682891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:06.683353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] PATH_INFO: `/v3/projects/4380b118062f4d2aac2fba5423a60807` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:06.687588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4380b118062f4d2aac2fba5423a60807)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:06.689783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:06.805884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4380b118062f4d2aac2fba5423a60807', 'request_id': 'req-e742f8f3-1134-4a5a-a038-fd6d325ac34b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:06.814988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4380b118062f4d2aac2fba5423a60807', 'request_id': 'req-e742f8f3-1134-4a5a-a038-fd6d325ac34b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:06.829991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4380b118062f4d2aac2fba5423a60807', 'request_id': 'req-e742f8f3-1134-4a5a-a038-fd6d325ac34b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:06.852064 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:06.857835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] The token cache is being invalidate because project 4380b118062f4d2aac2fba5423a60807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:06.858617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:06.859612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e742f8f3-1134-4a5a-a038-fd6d325ac34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:06.862151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 892/1774] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:06 2022] DELETE /identity/v3/projects/4380b118062f4d2aac2fba5423a60807 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:06.868007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6de30-3b55-45cd-ba2e-b49557a7aca2 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:06.868306 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6de30-3b55-45cd-ba2e-b49557a7aca2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:06.868591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6de30-3b55-45cd-ba2e-b49557a7aca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:06.918843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-18e6de30-3b55-45cd-ba2e-b49557a7aca2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:06.942782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-18e6de30-3b55-45cd-ba2e-b49557a7aca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:07.042253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 883/1775] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:07.051244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0668b7-d80e-4180-a618-e2603d3080ee None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:07.051538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0668b7-d80e-4180-a618-e2603d3080ee None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:07.051871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0668b7-d80e-4180-a618-e2603d3080ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:07.076025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6d0668b7-d80e-4180-a618-e2603d3080ee None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:07.084187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6d0668b7-d80e-4180-a618-e2603d3080ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:07.105171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 893/1776] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:07.111606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6de30-3b55-45cd-ba2e-b49557a7aca2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:07.128975 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6de30-3b55-45cd-ba2e-b49557a7aca2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:07.137992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feace95-9345-4959-ae52-1d39d426aee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:07.138706 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feace95-9345-4959-ae52-1d39d426aee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:07.139178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feace95-9345-4959-ae52-1d39d426aee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:07.139623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feace95-9345-4959-ae52-1d39d426aee0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:07.140525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feace95-9345-4959-ae52-1d39d426aee0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:07.141029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feace95-9345-4959-ae52-1d39d426aee0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:07.142938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feace95-9345-4959-ae52-1d39d426aee0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:07.163651 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0668b7-d80e-4180-a618-e2603d3080ee None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:07.172060 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 884/1777] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:27:07.176225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0668b7-d80e-4180-a618-e2603d3080ee None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:07.182884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b132c05-3ea0-4dc4-a167-fef8f9dfb303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:07.184281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b132c05-3ea0-4dc4-a167-fef8f9dfb303 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:07.184645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b132c05-3ea0-4dc4-a167-fef8f9dfb303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:07.184996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b132c05-3ea0-4dc4-a167-fef8f9dfb303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:07.185863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b132c05-3ea0-4dc4-a167-fef8f9dfb303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:07.189434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b132c05-3ea0-4dc4-a167-fef8f9dfb303 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:07.238815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feace95-9345-4959-ae52-1d39d426aee0 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:07.250054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 894/1778] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:07 2022] POST /identity/v3/projects => generated 381 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:07.259584 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feace95-9345-4959-ae52-1d39d426aee0 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:07.307463 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:07.404679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6f1a4-c48e-4491-b3d4-bcf775ba963b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:07.405816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6f1a4-c48e-4491-b3d4-bcf775ba963b admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:07.406891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6f1a4-c48e-4491-b3d4-bcf775ba963b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:07.408587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6f1a4-c48e-4491-b3d4-bcf775ba963b admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:07.409766 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6f1a4-c48e-4491-b3d4-bcf775ba963b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:07.416415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6f1a4-c48e-4491-b3d4-bcf775ba963b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:07.445349 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 885/1779] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:07 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:27:07.449088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b132c05-3ea0-4dc4-a167-fef8f9dfb303 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:07.469852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b132c05-3ea0-4dc4-a167-fef8f9dfb303 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:07.493814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db932ef1-25e6-4ae6-b39e-cc8b39f0345b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:07.496110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db932ef1-25e6-4ae6-b39e-cc8b39f0345b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:07.497623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db932ef1-25e6-4ae6-b39e-cc8b39f0345b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:07.497920 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db932ef1-25e6-4ae6-b39e-cc8b39f0345b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:07.505015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db932ef1-25e6-4ae6-b39e-cc8b39f0345b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:07.506921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db932ef1-25e6-4ae6-b39e-cc8b39f0345b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:07.538212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6f1a4-c48e-4491-b3d4-bcf775ba963b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:07.553231 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 895/1780] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:07.553490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6f1a4-c48e-4491-b3d4-bcf775ba963b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:07.566180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a7154-e3a8-4f6c-90ce-ad9e1f4d7dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:07.566860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a7154-e3a8-4f6c-90ce-ad9e1f4d7dce admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:07.569361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a7154-e3a8-4f6c-90ce-ad9e1f4d7dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:07.569820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a7154-e3a8-4f6c-90ce-ad9e1f4d7dce admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:07.571594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a7154-e3a8-4f6c-90ce-ad9e1f4d7dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:07.576825 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a7154-e3a8-4f6c-90ce-ad9e1f4d7dce admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:07.632319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db932ef1-25e6-4ae6-b39e-cc8b39f0345b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:07.633638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 886/1781] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:07.651503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db932ef1-25e6-4ae6-b39e-cc8b39f0345b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:07.662821 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02337c54-c5d2-4af2-93fc-fbdd5a6e3537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:07.665478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02337c54-c5d2-4af2-93fc-fbdd5a6e3537 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:07.665775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02337c54-c5d2-4af2-93fc-fbdd5a6e3537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:07.666120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02337c54-c5d2-4af2-93fc-fbdd5a6e3537 admin admin] PATH_INFO: `/v3/projects/99d2ca215cc8433881792e69e5e46734/users/425b4b16f7714e1abecc2880cce03d7d/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:07.745077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02337c54-c5d2-4af2-93fc-fbdd5a6e3537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d2ca215cc8433881792e69e5e46734, user_id=425b4b16f7714e1abecc2880cce03d7d, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:07.749959 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02337c54-c5d2-4af2-93fc-fbdd5a6e3537 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:07.772099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 896/1782] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:07 2022] PUT /identity/v3/projects/99d2ca215cc8433881792e69e5e46734/users/425b4b16f7714e1abecc2880cce03d7d/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:07.783106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac956a-0cff-476e-b4cd-f484fe8d33a2 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:07.783439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac956a-0cff-476e-b4cd-f484fe8d33a2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:07.783666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac956a-0cff-476e-b4cd-f484fe8d33a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:07.842158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e3ac956a-0cff-476e-b4cd-f484fe8d33a2 None None] MFA Rules not processed for user `425b4b16f7714e1abecc2880cce03d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:07.843540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e3ac956a-0cff-476e-b4cd-f484fe8d33a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:08.040440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 887/1783] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:08.053476 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02337c54-c5d2-4af2-93fc-fbdd5a6e3537 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:08.134057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02337c54-c5d2-4af2-93fc-fbdd5a6e3537 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:08.140071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63256558-1db8-4b9d-9804-b6c1a2499b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:08.140767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63256558-1db8-4b9d-9804-b6c1a2499b01 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:08.141109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63256558-1db8-4b9d-9804-b6c1a2499b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:08.141395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63256558-1db8-4b9d-9804-b6c1a2499b01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:08.142510 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63256558-1db8-4b9d-9804-b6c1a2499b01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:08.144539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63256558-1db8-4b9d-9804-b6c1a2499b01 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:08.194065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 897/1784] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:08 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:08.199168 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac956a-0cff-476e-b4cd-f484fe8d33a2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:08.237298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac956a-0cff-476e-b4cd-f484fe8d33a2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:08.374388 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa149625-6ebd-449c-bfbe-69e8c28c8f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:08.375025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa149625-6ebd-449c-bfbe-69e8c28c8f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:08.375374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa149625-6ebd-449c-bfbe-69e8c28c8f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:08.375724 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa149625-6ebd-449c-bfbe-69e8c28c8f27 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:08.376432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa149625-6ebd-449c-bfbe-69e8c28c8f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:08.378484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa149625-6ebd-449c-bfbe-69e8c28c8f27 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:08.444524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd689908-87ed-4601-8126-b9a161b08e80 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:08.444815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd689908-87ed-4601-8126-b9a161b08e80 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:08.445096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd689908-87ed-4601-8126-b9a161b08e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:08.448650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 888/1785] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:08 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:08.452908 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa149625-6ebd-449c-bfbe-69e8c28c8f27 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:08.463879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa149625-6ebd-449c-bfbe-69e8c28c8f27 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:08.466494 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-dd689908-87ed-4601-8126-b9a161b08e80 None None] MFA Rules not processed for user `47d7587d10b3487688ed251a85b83c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:08.468023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-dd689908-87ed-4601-8126-b9a161b08e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:08.470127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1c547-3e0f-4b9c-8972-bb37c151878d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:08.470955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1c547-3e0f-4b9c-8972-bb37c151878d admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:08.471320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1c547-3e0f-4b9c-8972-bb37c151878d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:08.471675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1c547-3e0f-4b9c-8972-bb37c151878d admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:08.472488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1c547-3e0f-4b9c-8972-bb37c151878d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:08.474352 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1c547-3e0f-4b9c-8972-bb37c151878d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:08.487000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 889/1786] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:08.491658 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1c547-3e0f-4b9c-8972-bb37c151878d admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:08.504581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1c547-3e0f-4b9c-8972-bb37c151878d admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:08.511096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee043da-5263-4e3d-931f-c653b68a1dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:08.512283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee043da-5263-4e3d-931f-c653b68a1dd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:08.512648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee043da-5263-4e3d-931f-c653b68a1dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:08.513973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee043da-5263-4e3d-931f-c653b68a1dd8 admin admin] PATH_INFO: `/v3/projects/b0fb8facd5074f1184198230bc79bf40/users/52c67d6710d2449c8da0b27faab6e04b/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:08.535760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee043da-5263-4e3d-931f-c653b68a1dd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0fb8facd5074f1184198230bc79bf40, user_id=52c67d6710d2449c8da0b27faab6e04b, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:08.538494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee043da-5263-4e3d-931f-c653b68a1dd8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:08.542512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:08.550679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 898/1787] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:08.557246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 890/1788] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:08 2022] PUT /identity/v3/projects/b0fb8facd5074f1184198230bc79bf40/users/52c67d6710d2449c8da0b27faab6e04b/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:08.561116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd689908-87ed-4601-8126-b9a161b08e80 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:08.574222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd689908-87ed-4601-8126-b9a161b08e80 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:08.617273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:08.681443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd24e2-43b8-436c-a6f8-11ef47a656c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:08.681443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd24e2-43b8-436c-a6f8-11ef47a656c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:08.681443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd24e2-43b8-436c-a6f8-11ef47a656c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:08.685497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd24e2-43b8-436c-a6f8-11ef47a656c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:08.686328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd24e2-43b8-436c-a6f8-11ef47a656c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:08.688014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd24e2-43b8-436c-a6f8-11ef47a656c9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:08.703687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 899/1789] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:08.707914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee043da-5263-4e3d-931f-c653b68a1dd8 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:08.721520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee043da-5263-4e3d-931f-c653b68a1dd8 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:08.728137 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700088e-3af0-4b7d-b36a-178fa3bddf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:08.729083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700088e-3af0-4b7d-b36a-178fa3bddf25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:08.729365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700088e-3af0-4b7d-b36a-178fa3bddf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:08.729642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700088e-3af0-4b7d-b36a-178fa3bddf25 admin admin] PATH_INFO: `/v3/projects/b0fb8facd5074f1184198230bc79bf40/users/52c67d6710d2449c8da0b27faab6e04b/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:08.732609 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700088e-3af0-4b7d-b36a-178fa3bddf25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0fb8facd5074f1184198230bc79bf40, user_id=52c67d6710d2449c8da0b27faab6e04b, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:08.736042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700088e-3af0-4b7d-b36a-178fa3bddf25 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:08.749196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 891/1790] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:08 2022] PUT /identity/v3/projects/b0fb8facd5074f1184198230bc79bf40/users/52c67d6710d2449c8da0b27faab6e04b/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:08.760158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4f619-91e0-451a-a268-80b80ebadd26 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:08.760897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4f619-91e0-451a-a268-80b80ebadd26 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:08.761205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4f619-91e0-451a-a268-80b80ebadd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:08.793608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8bf4f619-91e0-451a-a268-80b80ebadd26 None None] MFA Rules not processed for user `52c67d6710d2449c8da0b27faab6e04b`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:08.794900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8bf4f619-91e0-451a-a268-80b80ebadd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:08.875427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 900/1791] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:08.887252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700088e-3af0-4b7d-b36a-178fa3bddf25 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:08.888948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b700088e-3af0-4b7d-b36a-178fa3bddf25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:08.961047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700088e-3af0-4b7d-b36a-178fa3bddf25 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:08.965219 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3100ac-3570-4823-94b2-55d3b6495dba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:08.965878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3100ac-3570-4823-94b2-55d3b6495dba service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:08.966159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3100ac-3570-4823-94b2-55d3b6495dba service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:08.966430 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3100ac-3570-4823-94b2-55d3b6495dba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:08.977252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3100ac-3570-4823-94b2-55d3b6495dba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:08.978742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3100ac-3570-4823-94b2-55d3b6495dba service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:08.990044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 892/1792] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:09.107857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4f619-91e0-451a-a268-80b80ebadd26 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:09.119601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4f619-91e0-451a-a268-80b80ebadd26 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:09.125398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f5931-9c0c-428e-be3e-f1b6ebcab571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:09.126113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f5931-9c0c-428e-be3e-f1b6ebcab571 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:09.127155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f5931-9c0c-428e-be3e-f1b6ebcab571 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:09.127622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f5931-9c0c-428e-be3e-f1b6ebcab571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:09.205292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f5931-9c0c-428e-be3e-f1b6ebcab571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:09.206926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f5931-9c0c-428e-be3e-f1b6ebcab571 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:09.217281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 901/1793] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:09.711982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3100ac-3570-4823-94b2-55d3b6495dba service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:09.833929 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3100ac-3570-4823-94b2-55d3b6495dba service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:09.844319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:09.845313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:09.846187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:09.846488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] PATH_INFO: `/v3/users/425b4b16f7714e1abecc2880cce03d7d` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:09.848312 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=425b4b16f7714e1abecc2880cce03d7d)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:09.884910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:09.988928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425b4b16f7714e1abecc2880cce03d7d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:10.009693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425b4b16f7714e1abecc2880cce03d7d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:10.016949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425b4b16f7714e1abecc2880cce03d7d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:10.042837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425b4b16f7714e1abecc2880cce03d7d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:10.048505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425b4b16f7714e1abecc2880cce03d7d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:10.098760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 893/1794] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:09 2022] DELETE /identity/v3/users/425b4b16f7714e1abecc2880cce03d7d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:10.104100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59f044-73e5-488f-859b-3ebb91b589d4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:10.104733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59f044-73e5-488f-859b-3ebb91b589d4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:10.104733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59f044-73e5-488f-859b-3ebb91b589d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:10.156408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bd59f044-73e5-488f-859b-3ebb91b589d4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:10.159099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bd59f044-73e5-488f-859b-3ebb91b589d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:10.344104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:10.401385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 902/1795] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:10.418032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:10.420875 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:10.435685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:10.612372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af18551-1de0-41ff-ae47-c341bc797833 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:10.616572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88b901-4889-4156-939a-324bed7e20e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:10.617804 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88b901-4889-4156-939a-324bed7e20e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:10.620910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88b901-4889-4156-939a-324bed7e20e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:10.620910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88b901-4889-4156-939a-324bed7e20e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:10.630722 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88b901-4889-4156-939a-324bed7e20e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:10.632224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88b901-4889-4156-939a-324bed7e20e1 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:10.643633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 894/1796] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:11.312400 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59f044-73e5-488f-859b-3ebb91b589d4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:11.314727 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bd59f044-73e5-488f-859b-3ebb91b589d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:11.351781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59f044-73e5-488f-859b-3ebb91b589d4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:11.362979 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:11.364661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:11.365175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:11.365652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] PATH_INFO: `/v3/projects/99d2ca215cc8433881792e69e5e46734` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:11.367269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] RBAC: Authorizing `identity:delete_project(project_id=99d2ca215cc8433881792e69e5e46734)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:11.370277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:11.482261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99d2ca215cc8433881792e69e5e46734', 'request_id': 'req-9d3edb8b-3d89-4edf-9033-15720187605a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:11.499462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99d2ca215cc8433881792e69e5e46734', 'request_id': 'req-9d3edb8b-3d89-4edf-9033-15720187605a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:11.523650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99d2ca215cc8433881792e69e5e46734', 'request_id': 'req-9d3edb8b-3d89-4edf-9033-15720187605a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:11.547985 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] The token cache is being invalidate because project 99d2ca215cc8433881792e69e5e46734 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:11.547985 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:11.549988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:11.555867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 903/1797] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:11 2022] DELETE /identity/v3/projects/99d2ca215cc8433881792e69e5e46734 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:11.556616 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88b901-4889-4156-939a-324bed7e20e1 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:11.578940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88b901-4889-4156-939a-324bed7e20e1 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:11.821971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e6d21-f846-4d2c-8cd2-3d38c82f28b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:11.823575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6d21-f846-4d2c-8cd2-3d38c82f28b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:11.823870 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6d21-f846-4d2c-8cd2-3d38c82f28b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:11.824153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e6d21-f846-4d2c-8cd2-3d38c82f28b4 admin admin] PATH_INFO: `/v3/users/52c67d6710d2449c8da0b27faab6e04b` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:11.827772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e6d21-f846-4d2c-8cd2-3d38c82f28b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c67d6710d2449c8da0b27faab6e04b)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:11.831580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7e6d21-f846-4d2c-8cd2-3d38c82f28b4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:11.951876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-de7e6d21-f846-4d2c-8cd2-3d38c82f28b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c67d6710d2449c8da0b27faab6e04b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:11.977889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-de7e6d21-f846-4d2c-8cd2-3d38c82f28b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c67d6710d2449c8da0b27faab6e04b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:11.979922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-de7e6d21-f846-4d2c-8cd2-3d38c82f28b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c67d6710d2449c8da0b27faab6e04b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:11.994464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-de7e6d21-f846-4d2c-8cd2-3d38c82f28b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c67d6710d2449c8da0b27faab6e04b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:11.996588 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-de7e6d21-f846-4d2c-8cd2-3d38c82f28b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c67d6710d2449c8da0b27faab6e04b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:12.024115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 895/1798] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:11 2022] DELETE /identity/v3/users/52c67d6710d2449c8da0b27faab6e04b => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:12.096959 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:12.098752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:12.128186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3edb8b-3d89-4edf-9033-15720187605a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:12.257441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75f851-682d-4ce8-8079-36b6093b8734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:12.259036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75f851-682d-4ce8-8079-36b6093b8734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:12.259322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75f851-682d-4ce8-8079-36b6093b8734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:12.259662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75f851-682d-4ce8-8079-36b6093b8734 admin admin] PATH_INFO: `/v3/users/0afa6ba21f64474b803463c1773848b9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:12.262678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75f851-682d-4ce8-8079-36b6093b8734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0afa6ba21f64474b803463c1773848b9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:12.265358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75f851-682d-4ce8-8079-36b6093b8734 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:12.356552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7b75f851-682d-4ce8-8079-36b6093b8734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0afa6ba21f64474b803463c1773848b9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:12.362143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217130b5-086d-4a40-8d0b-80ef02d8edb7 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:12.362354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217130b5-086d-4a40-8d0b-80ef02d8edb7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:12.362604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217130b5-086d-4a40-8d0b-80ef02d8edb7 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:12.366016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 896/1799] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:27:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 03 15:27:12.376400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ea26c-24c6-4643-9b21-8b2ac3bc49ed None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:12.376726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ea26c-24c6-4643-9b21-8b2ac3bc49ed None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:12.376973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ea26c-24c6-4643-9b21-8b2ac3bc49ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:12.413320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7b75f851-682d-4ce8-8079-36b6093b8734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0afa6ba21f64474b803463c1773848b9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:12.415516 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7b75f851-682d-4ce8-8079-36b6093b8734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0afa6ba21f64474b803463c1773848b9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:12.432967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-6d8ea26c-24c6-4643-9b21-8b2ac3bc49ed None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:12.435588 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6d8ea26c-24c6-4643-9b21-8b2ac3bc49ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:12.436489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:12.440076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7b75f851-682d-4ce8-8079-36b6093b8734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0afa6ba21f64474b803463c1773848b9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:12.441156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7b75f851-682d-4ce8-8079-36b6093b8734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0afa6ba21f64474b803463c1773848b9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:12.469747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 904/1800] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:12 2022] DELETE /identity/v3/users/0afa6ba21f64474b803463c1773848b9 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:12.471581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740820d1-ce2c-494b-a6af-8cf06381e7f2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:12.471880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740820d1-ce2c-494b-a6af-8cf06381e7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:12.472163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740820d1-ce2c-494b-a6af-8cf06381e7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:12.482597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:12.509632 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-740820d1-ce2c-494b-a6af-8cf06381e7f2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:12.510462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-740820d1-ce2c-494b-a6af-8cf06381e7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:12.593144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 897/1801] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:27:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:12.610011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ea26c-24c6-4643-9b21-8b2ac3bc49ed None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:12.611502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6d8ea26c-24c6-4643-9b21-8b2ac3bc49ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:12.628330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ea26c-24c6-4643-9b21-8b2ac3bc49ed None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:12.637446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 905/1802] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:12.648438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740820d1-ce2c-494b-a6af-8cf06381e7f2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:12.650329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-740820d1-ce2c-494b-a6af-8cf06381e7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:12.763010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f318290-16a5-4f26-bdca-86805fba6900 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:12.764233 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f318290-16a5-4f26-bdca-86805fba6900 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:12.765268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f318290-16a5-4f26-bdca-86805fba6900 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:12.765697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f318290-16a5-4f26-bdca-86805fba6900 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:12.770349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740820d1-ce2c-494b-a6af-8cf06381e7f2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:12.781246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f318290-16a5-4f26-bdca-86805fba6900 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:12.790188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f318290-16a5-4f26-bdca-86805fba6900 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:12.795081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca2e9c-fb89-4a3e-8dae-3f7327e90a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:12.797630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca2e9c-fb89-4a3e-8dae-3f7327e90a25 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:12.798109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca2e9c-fb89-4a3e-8dae-3f7327e90a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:12.798769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca2e9c-fb89-4a3e-8dae-3f7327e90a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:12.810869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 898/1803] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:27:12.823022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca2e9c-fb89-4a3e-8dae-3f7327e90a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:12.825281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca2e9c-fb89-4a3e-8dae-3f7327e90a25 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:12.840802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 906/1804] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:27:13.166571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f318290-16a5-4f26-bdca-86805fba6900 service nova] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:13.169337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7f318290-16a5-4f26-bdca-86805fba6900 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:13.203482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f318290-16a5-4f26-bdca-86805fba6900 service nova] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:13.220278 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:13.221199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:13.221486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:13.221761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] PATH_INFO: `/v3/projects/b0fb8facd5074f1184198230bc79bf40` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:13.223208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0fb8facd5074f1184198230bc79bf40)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:13.232546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:13.332719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0fb8facd5074f1184198230bc79bf40', 'request_id': 'req-40ed54d5-01fa-410e-ab48-c41b0ae808b6'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.358368 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0fb8facd5074f1184198230bc79bf40', 'request_id': 'req-40ed54d5-01fa-410e-ab48-c41b0ae808b6'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.375945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0fb8facd5074f1184198230bc79bf40', 'request_id': 'req-40ed54d5-01fa-410e-ab48-c41b0ae808b6'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.416946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] The token cache is being invalidate because project b0fb8facd5074f1184198230bc79bf40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:13.417307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.418223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-40ed54d5-01fa-410e-ab48-c41b0ae808b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.422605 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 899/1805] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:13 2022] DELETE /identity/v3/projects/b0fb8facd5074f1184198230bc79bf40 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:13.425736 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6293d70-5600-4d3f-a19f-8cc2caf6a0cd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:13.426401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6293d70-5600-4d3f-a19f-8cc2caf6a0cd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:13.426401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6293d70-5600-4d3f-a19f-8cc2caf6a0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:13.448120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a6293d70-5600-4d3f-a19f-8cc2caf6a0cd None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:13.450630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a6293d70-5600-4d3f-a19f-8cc2caf6a0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:13.518732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 907/1806] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:13.527532 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a143-fcf6-4196-a4f4-ad3127d7450d None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:13.527817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a143-fcf6-4196-a4f4-ad3127d7450d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:13.528099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1030a143-fcf6-4196-a4f4-ad3127d7450d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:13.546518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-1030a143-fcf6-4196-a4f4-ad3127d7450d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:13.547845 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1030a143-fcf6-4196-a4f4-ad3127d7450d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:13.556710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 900/1807] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:13.560720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6293d70-5600-4d3f-a19f-8cc2caf6a0cd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:13.596378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6293d70-5600-4d3f-a19f-8cc2caf6a0cd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:13.602897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfea43-5a23-4271-a7ef-6ca74d02cd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:13.603578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfea43-5a23-4271-a7ef-6ca74d02cd77 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:13.603864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfea43-5a23-4271-a7ef-6ca74d02cd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:13.604139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfea43-5a23-4271-a7ef-6ca74d02cd77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:13.604998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfea43-5a23-4271-a7ef-6ca74d02cd77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:13.605405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfea43-5a23-4271-a7ef-6ca74d02cd77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:13.607534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfea43-5a23-4271-a7ef-6ca74d02cd77 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:13.623141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030a143-fcf6-4196-a4f4-ad3127d7450d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:13.626237 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1030a143-fcf6-4196-a4f4-ad3127d7450d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:13.631584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 908/1808] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:27:13.633666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfea43-5a23-4271-a7ef-6ca74d02cd77 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:13.656024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfea43-5a23-4271-a7ef-6ca74d02cd77 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:13.661654 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcbadb-87b9-4e3f-9744-de81127503a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:13.663164 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcbadb-87b9-4e3f-9744-de81127503a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:13.663531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcbadb-87b9-4e3f-9744-de81127503a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:13.664252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcbadb-87b9-4e3f-9744-de81127503a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:13.664984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcbadb-87b9-4e3f-9744-de81127503a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:13.666831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1030a143-fcf6-4196-a4f4-ad3127d7450d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:13.667024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcbadb-87b9-4e3f-9744-de81127503a4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:13.672853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:13.673741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:13.674026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:13.674477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] PATH_INFO: `/v3/projects/53e1947047fb4658bc271ae0ddf85cb3` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:13.675765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e1947047fb4658bc271ae0ddf85cb3)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:13.678448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:13.693816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:13.700242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 909/1809] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:13 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:13.700427 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:13.703702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcbadb-87b9-4e3f-9744-de81127503a4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:13.741709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e1947047fb4658bc271ae0ddf85cb3', 'request_id': 'req-f957cfb0-e287-4b3d-93e7-368cb552f163'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.752386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e1947047fb4658bc271ae0ddf85cb3', 'request_id': 'req-f957cfb0-e287-4b3d-93e7-368cb552f163'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.760877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e1947047fb4658bc271ae0ddf85cb3', 'request_id': 'req-f957cfb0-e287-4b3d-93e7-368cb552f163'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.773663 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] The token cache is being invalidate because project 53e1947047fb4658bc271ae0ddf85cb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:13.773960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.774662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.776794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 901/1810] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:13 2022] DELETE /identity/v3/projects/53e1947047fb4658bc271ae0ddf85cb3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:13.780785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:13.781760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcbadb-87b9-4e3f-9744-de81127503a4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:13.782316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:13.854700 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa8185-5de1-4d21-978c-6becc3171cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:13.855385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa8185-5de1-4d21-978c-6becc3171cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:13.855725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa8185-5de1-4d21-978c-6becc3171cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:13.856084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa8185-5de1-4d21-978c-6becc3171cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:13.856796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa8185-5de1-4d21-978c-6becc3171cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:13.857056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f957cfb0-e287-4b3d-93e7-368cb552f163 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:13.858846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa8185-5de1-4d21-978c-6becc3171cc8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:13.862171 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6e3b6-9221-46dc-8e8f-5074b967aa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:13.863084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6e3b6-9221-46dc-8e8f-5074b967aa4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:13.863391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6e3b6-9221-46dc-8e8f-5074b967aa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:13.863667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6e3b6-9221-46dc-8e8f-5074b967aa4d admin admin] PATH_INFO: `/v3/users/f8cd031247704e8796ea8b6534e2610d` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:13.864953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6e3b6-9221-46dc-8e8f-5074b967aa4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8cd031247704e8796ea8b6534e2610d)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:13.866886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6e3b6-9221-46dc-8e8f-5074b967aa4d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:13.882731 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 910/1811] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:13 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:27:13.887001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa8185-5de1-4d21-978c-6becc3171cc8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:13.888689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cfaa8185-5de1-4d21-978c-6becc3171cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:13.904277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa8185-5de1-4d21-978c-6becc3171cc8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:13.909924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545e0e2-b593-4b39-8b98-bb095abb9109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:13.911050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545e0e2-b593-4b39-8b98-bb095abb9109 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:13.911342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545e0e2-b593-4b39-8b98-bb095abb9109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:13.911622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545e0e2-b593-4b39-8b98-bb095abb9109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:13.912355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545e0e2-b593-4b39-8b98-bb095abb9109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:13.914569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545e0e2-b593-4b39-8b98-bb095abb9109 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:13.925962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 911/1812] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:27:13.928576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f8d6e3b6-9221-46dc-8e8f-5074b967aa4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8cd031247704e8796ea8b6534e2610d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.930797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545e0e2-b593-4b39-8b98-bb095abb9109 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:13.941989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545e0e2-b593-4b39-8b98-bb095abb9109 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:13.944044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f8d6e3b6-9221-46dc-8e8f-5074b967aa4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8cd031247704e8796ea8b6534e2610d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.946169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f8d6e3b6-9221-46dc-8e8f-5074b967aa4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8cd031247704e8796ea8b6534e2610d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.948041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ecab4-3ea8-47ce-bb71-ad990eee35f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:13.948793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ecab4-3ea8-47ce-bb71-ad990eee35f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:13.949106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ecab4-3ea8-47ce-bb71-ad990eee35f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:13.949357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ecab4-3ea8-47ce-bb71-ad990eee35f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:13.950097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ecab4-3ea8-47ce-bb71-ad990eee35f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:13.951799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ecab4-3ea8-47ce-bb71-ad990eee35f7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:13.955230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f8d6e3b6-9221-46dc-8e8f-5074b967aa4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8cd031247704e8796ea8b6534e2610d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.956063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f8d6e3b6-9221-46dc-8e8f-5074b967aa4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8cd031247704e8796ea8b6534e2610d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:13.966109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 912/1813] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:13.969503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ecab4-3ea8-47ce-bb71-ad990eee35f7 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:13.971363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a55ecab4-3ea8-47ce-bb71-ad990eee35f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:13.976121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 902/1814] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:13 2022] DELETE /identity/v3/users/f8cd031247704e8796ea8b6534e2610d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:13.985287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ecab4-3ea8-47ce-bb71-ad990eee35f7 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:14.061456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db29df3-09d0-4e9f-9998-f4ff4f6ede6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:14.062400 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29df3-09d0-4e9f-9998-f4ff4f6ede6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:14.062713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29df3-09d0-4e9f-9998-f4ff4f6ede6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:14.063140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29df3-09d0-4e9f-9998-f4ff4f6ede6f admin admin] PATH_INFO: `/v3/projects/af6882ed2e30401683064e262659ef63/users/66a20306e93547969ff205e1da22b5f0/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:14.079340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db29df3-09d0-4e9f-9998-f4ff4f6ede6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=af6882ed2e30401683064e262659ef63, user_id=66a20306e93547969ff205e1da22b5f0, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:14.082086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db29df3-09d0-4e9f-9998-f4ff4f6ede6f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:14.085430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:14.100423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 913/1815] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:13 2022] PUT /identity/v3/projects/af6882ed2e30401683064e262659ef63/users/66a20306e93547969ff205e1da22b5f0/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:14.112234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493e591-e4c4-45d5-871b-6c30812b6787 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:14.112418 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493e591-e4c4-45d5-871b-6c30812b6787 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:14.112810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493e591-e4c4-45d5-871b-6c30812b6787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:14.146908 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-3493e591-e4c4-45d5-871b-6c30812b6787 None None] MFA Rules not processed for user `66a20306e93547969ff205e1da22b5f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:14.148982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3493e591-e4c4-45d5-871b-6c30812b6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:14.170916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:14.239165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 903/1816] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:14.250773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db29df3-09d0-4e9f-9998-f4ff4f6ede6f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:14.253076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5db29df3-09d0-4e9f-9998-f4ff4f6ede6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:14.281057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493e591-e4c4-45d5-871b-6c30812b6787 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:14.284241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3493e591-e4c4-45d5-871b-6c30812b6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:14.344143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db29df3-09d0-4e9f-9998-f4ff4f6ede6f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:14.349057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7e07f-c849-40e6-99a3-e45f29920c57 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13121a263c31433dbd4a5082b4bf3bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:14.350071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e07f-c849-40e6-99a3-e45f29920c57 service glance] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:14.350416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e07f-c849-40e6-99a3-e45f29920c57 service glance] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:14.350824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e07f-c849-40e6-99a3-e45f29920c57 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:14.365978 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7e07f-c849-40e6-99a3-e45f29920c57 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:14.369384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7e07f-c849-40e6-99a3-e45f29920c57 service glance] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:14.387817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 914/1817] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:14.398935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493e591-e4c4-45d5-871b-6c30812b6787 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:14.405796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:14.407660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:14.407942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:14.408217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] PATH_INFO: `/v3/projects/d076ac6ad92e4732886152da39a14f32` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:14.412197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d076ac6ad92e4732886152da39a14f32)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:14.414820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:14.479803 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd076ac6ad92e4732886152da39a14f32', 'request_id': 'req-7bf8cb60-7f35-4eec-93e1-c618570f4d67'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:14.491280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd076ac6ad92e4732886152da39a14f32', 'request_id': 'req-7bf8cb60-7f35-4eec-93e1-c618570f4d67'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:14.505129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd076ac6ad92e4732886152da39a14f32', 'request_id': 'req-7bf8cb60-7f35-4eec-93e1-c618570f4d67'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:14.527144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] The token cache is being invalidate because project d076ac6ad92e4732886152da39a14f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:14.527445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:14.528318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7bf8cb60-7f35-4eec-93e1-c618570f4d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:14.530423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 904/1818] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:14 2022] DELETE /identity/v3/projects/d076ac6ad92e4732886152da39a14f32 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:14.535185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263453b-44c2-4610-af25-13273d5e48f7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:14.535625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263453b-44c2-4610-af25-13273d5e48f7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:14.535750 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263453b-44c2-4610-af25-13273d5e48f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:14.555464 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-4263453b-44c2-4610-af25-13273d5e48f7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:14.556486 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4263453b-44c2-4610-af25-13273d5e48f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:14.650505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 915/1819] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:14.658633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759486-3ffa-4589-801c-ab17b9555bbe None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:14.659160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759486-3ffa-4589-801c-ab17b9555bbe None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:14.659261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759486-3ffa-4589-801c-ab17b9555bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:14.684849 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-76759486-3ffa-4589-801c-ab17b9555bbe None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:14.685476 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-76759486-3ffa-4589-801c-ab17b9555bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:14.699288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 905/1820] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:14.704297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263453b-44c2-4610-af25-13273d5e48f7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:14.715007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263453b-44c2-4610-af25-13273d5e48f7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:14.720127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c155aa-8f74-4bce-8bf2-2099d488bed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:14.720956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c155aa-8f74-4bce-8bf2-2099d488bed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:14.721323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c155aa-8f74-4bce-8bf2-2099d488bed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:14.721632 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c155aa-8f74-4bce-8bf2-2099d488bed7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:14.722362 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c155aa-8f74-4bce-8bf2-2099d488bed7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:14.722711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c155aa-8f74-4bce-8bf2-2099d488bed7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:14.724337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c155aa-8f74-4bce-8bf2-2099d488bed7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:14.737549 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 916/1821] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:27:14.742102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759486-3ffa-4589-801c-ab17b9555bbe None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:14.752324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759486-3ffa-4589-801c-ab17b9555bbe None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:14.756995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca499de4-ea1c-4649-8701-6392e34c2b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:14.758188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca499de4-ea1c-4649-8701-6392e34c2b6c admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:14.758474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca499de4-ea1c-4649-8701-6392e34c2b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:14.758797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca499de4-ea1c-4649-8701-6392e34c2b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:14.760153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca499de4-ea1c-4649-8701-6392e34c2b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:14.762101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca499de4-ea1c-4649-8701-6392e34c2b6c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:14.794790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 906/1822] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:14 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:27:14.796397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c155aa-8f74-4bce-8bf2-2099d488bed7 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:14.812302 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c155aa-8f74-4bce-8bf2-2099d488bed7 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:14.890444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba4eb9-12e6-4ce2-b8e8-2cf18868b731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:14.891314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba4eb9-12e6-4ce2-b8e8-2cf18868b731 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:14.891732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba4eb9-12e6-4ce2-b8e8-2cf18868b731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:14.892150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba4eb9-12e6-4ce2-b8e8-2cf18868b731 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:14.892718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba4eb9-12e6-4ce2-b8e8-2cf18868b731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:14.894848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba4eb9-12e6-4ce2-b8e8-2cf18868b731 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:14.939889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 917/1823] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:14 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:14.944129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca499de4-ea1c-4649-8701-6392e34c2b6c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:14.963788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca499de4-ea1c-4649-8701-6392e34c2b6c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:14.991276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca83c5f-06d9-4f05-adf1-f3dd8cc28421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:14.992514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca83c5f-06d9-4f05-adf1-f3dd8cc28421 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:14.992797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca83c5f-06d9-4f05-adf1-f3dd8cc28421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:14.993243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca83c5f-06d9-4f05-adf1-f3dd8cc28421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:14.993990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca83c5f-06d9-4f05-adf1-f3dd8cc28421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:14.995720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca83c5f-06d9-4f05-adf1-f3dd8cc28421 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:15.034172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 907/1824] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:15.038751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba4eb9-12e6-4ce2-b8e8-2cf18868b731 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:15.062374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba4eb9-12e6-4ce2-b8e8-2cf18868b731 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:15.075085 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbb32c-f500-41de-bf85-2c2784ffae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:15.075820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbb32c-f500-41de-bf85-2c2784ffae71 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:15.076184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbb32c-f500-41de-bf85-2c2784ffae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:15.076560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbb32c-f500-41de-bf85-2c2784ffae71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:15.077413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbb32c-f500-41de-bf85-2c2784ffae71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:15.079198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbb32c-f500-41de-bf85-2c2784ffae71 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:15.105327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca83c5f-06d9-4f05-adf1-f3dd8cc28421 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:15.107416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 918/1825] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:15.112778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:15.128422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca83c5f-06d9-4f05-adf1-f3dd8cc28421 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:15.139467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ffc1b-30df-4131-8ddc-1c5cce8d0fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:15.142511 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ffc1b-30df-4131-8ddc-1c5cce8d0fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:15.142939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ffc1b-30df-4131-8ddc-1c5cce8d0fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:15.143738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ffc1b-30df-4131-8ddc-1c5cce8d0fc4 admin admin] PATH_INFO: `/v3/projects/5e123f382d3f4486a16d21be0cf3fc35/users/010d037d19fa4d9d9333c44b1796d81c/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:15.168031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ffc1b-30df-4131-8ddc-1c5cce8d0fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e123f382d3f4486a16d21be0cf3fc35, user_id=010d037d19fa4d9d9333c44b1796d81c, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:15.170941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ffc1b-30df-4131-8ddc-1c5cce8d0fc4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:15.187576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 908/1826] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:15 2022] PUT /identity/v3/projects/5e123f382d3f4486a16d21be0cf3fc35/users/010d037d19fa4d9d9333c44b1796d81c/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:15.199661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed823448-ec6d-489d-a3a4-6f18a047585d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:15.199950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed823448-ec6d-489d-a3a4-6f18a047585d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:15.200416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed823448-ec6d-489d-a3a4-6f18a047585d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:15.207623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:15.239925 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ed823448-ec6d-489d-a3a4-6f18a047585d None None] MFA Rules not processed for user `010d037d19fa4d9d9333c44b1796d81c`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:15.242864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ed823448-ec6d-489d-a3a4-6f18a047585d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:15.370468 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 919/1827] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:15.378180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ffc1b-30df-4131-8ddc-1c5cce8d0fc4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:15.380051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-112ffc1b-30df-4131-8ddc-1c5cce8d0fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:15.582822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ffc1b-30df-4131-8ddc-1c5cce8d0fc4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:15.591043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2483b4ac-9b86-457e-8241-5417306de817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:15.591922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483b4ac-9b86-457e-8241-5417306de817 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:15.592351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483b4ac-9b86-457e-8241-5417306de817 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:15.592732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2483b4ac-9b86-457e-8241-5417306de817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:15.603918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483b4ac-9b86-457e-8241-5417306de817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:15.606608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2483b4ac-9b86-457e-8241-5417306de817 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:15.631985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 909/1828] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:15.667260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed823448-ec6d-489d-a3a4-6f18a047585d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:15.798826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed823448-ec6d-489d-a3a4-6f18a047585d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:15.805373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:15.806697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:15.807096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:15.807616 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] PATH_INFO: `/v3/users/010d037d19fa4d9d9333c44b1796d81c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:15.810839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=010d037d19fa4d9d9333c44b1796d81c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:15.813228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:15.925198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010d037d19fa4d9d9333c44b1796d81c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:15.951383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010d037d19fa4d9d9333c44b1796d81c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:15.952551 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010d037d19fa4d9d9333c44b1796d81c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:15.963959 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010d037d19fa4d9d9333c44b1796d81c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:15.964947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010d037d19fa4d9d9333c44b1796d81c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:15.993634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 920/1829] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:15 2022] DELETE /identity/v3/users/010d037d19fa4d9d9333c44b1796d81c => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:15.998961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa008d-4471-4cd5-8c55-e7670b7050d0 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:15.999645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa008d-4471-4cd5-8c55-e7670b7050d0 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:15.999805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa008d-4471-4cd5-8c55-e7670b7050d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:16.067586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d8aa008d-4471-4cd5-8c55-e7670b7050d0 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:16.068853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d8aa008d-4471-4cd5-8c55-e7670b7050d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:16.249823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 910/1830] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:27:16.306026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:16.306791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:16.331174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:16.418213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7ed62c-8024-4188-bdc6-65f552243cdc admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:16.423283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d319c-41a0-49de-8cd5-60794f1c7c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:16.423283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d319c-41a0-49de-8cd5-60794f1c7c9d service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:16.425190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d319c-41a0-49de-8cd5-60794f1c7c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:16.425470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d319c-41a0-49de-8cd5-60794f1c7c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:16.459416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d319c-41a0-49de-8cd5-60794f1c7c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:16.461180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d319c-41a0-49de-8cd5-60794f1c7c9d service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:16.496597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 921/1831] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:17.899482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2298-a7e8-45e2-9dbf-8e52d7438220 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:17.900474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2298-a7e8-45e2-9dbf-8e52d7438220 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:17.900474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2298-a7e8-45e2-9dbf-8e52d7438220 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:17.901422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 911/1832] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:27:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:27:17.915871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d319c-41a0-49de-8cd5-60794f1c7c9d service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:17.917276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3e9d319c-41a0-49de-8cd5-60794f1c7c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:17.943892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d319c-41a0-49de-8cd5-60794f1c7c9d service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:17.963928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:17.965628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:17.966126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:17.966666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] PATH_INFO: `/v3/projects/5e123f382d3f4486a16d21be0cf3fc35` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:17.969953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e123f382d3f4486a16d21be0cf3fc35)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:17.971911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3bbd0-f26a-45c2-86a3-056ab00bc40e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:17.972174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3bbd0-f26a-45c2-86a3-056ab00bc40e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:17.972480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3bbd0-f26a-45c2-86a3-056ab00bc40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:17.977079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:18.026601 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-f4b3bbd0-f26a-45c2-86a3-056ab00bc40e None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:18.028888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f4b3bbd0-f26a-45c2-86a3-056ab00bc40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:18.189708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e123f382d3f4486a16d21be0cf3fc35', 'request_id': 'req-57d8f60d-411f-4f61-abb4-a703e90e43cf'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:18.209811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e123f382d3f4486a16d21be0cf3fc35', 'request_id': 'req-57d8f60d-411f-4f61-abb4-a703e90e43cf'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:18.248488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e123f382d3f4486a16d21be0cf3fc35', 'request_id': 'req-57d8f60d-411f-4f61-abb4-a703e90e43cf'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:18.273127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] The token cache is being invalidate because project 5e123f382d3f4486a16d21be0cf3fc35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:18.278543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:18.280396 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 912/1833] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:27:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:27:18.280768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-57d8f60d-411f-4f61-abb4-a703e90e43cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:18.283975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 922/1834] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:17 2022] DELETE /identity/v3/projects/5e123f382d3f4486a16d21be0cf3fc35 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:18.293093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3482d56-f406-4bd0-bdc6-ac37aed8070f None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:18.293093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3482d56-f406-4bd0-bdc6-ac37aed8070f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:18.293093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3482d56-f406-4bd0-bdc6-ac37aed8070f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:18.341317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d3482d56-f406-4bd0-bdc6-ac37aed8070f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:18.342820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d3482d56-f406-4bd0-bdc6-ac37aed8070f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:18.360273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:18.518598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 913/1835] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:18.520348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f16b5-7657-4ffa-a32d-1d9a6b8c8669 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:18.520636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f16b5-7657-4ffa-a32d-1d9a6b8c8669 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:18.520913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f16b5-7657-4ffa-a32d-1d9a6b8c8669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:18.553146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f11f16b5-7657-4ffa-a32d-1d9a6b8c8669 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:18.555538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f11f16b5-7657-4ffa-a32d-1d9a6b8c8669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:18.592667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 923/1836] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:18.594431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3482d56-f406-4bd0-bdc6-ac37aed8070f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:18.604783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3482d56-f406-4bd0-bdc6-ac37aed8070f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:18.620464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3937e2a-096c-4deb-ab14-b7869d785dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:18.621518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3937e2a-096c-4deb-ab14-b7869d785dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:18.622036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3937e2a-096c-4deb-ab14-b7869d785dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:18.636459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3937e2a-096c-4deb-ab14-b7869d785dd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:18.636459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3937e2a-096c-4deb-ab14-b7869d785dd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:18.636459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3937e2a-096c-4deb-ab14-b7869d785dd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:18.636459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3937e2a-096c-4deb-ab14-b7869d785dd9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:18.669431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 914/1837] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:27:18.677308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f16b5-7657-4ffa-a32d-1d9a6b8c8669 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:18.695964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f16b5-7657-4ffa-a32d-1d9a6b8c8669 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:18.703709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e952059-0065-497c-80ab-58a80ba242a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:18.704432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e952059-0065-497c-80ab-58a80ba242a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:18.704710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e952059-0065-497c-80ab-58a80ba242a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:18.704982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e952059-0065-497c-80ab-58a80ba242a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:18.706033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e952059-0065-497c-80ab-58a80ba242a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:18.708042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e952059-0065-497c-80ab-58a80ba242a9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:18.753037 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 924/1838] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:18 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:18.754158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3937e2a-096c-4deb-ab14-b7869d785dd9 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:18.771796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3937e2a-096c-4deb-ab14-b7869d785dd9 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:18.870319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d36e15-52b1-4dc6-9473-69ae1347a2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:18.871961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d36e15-52b1-4dc6-9473-69ae1347a2f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:18.872335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d36e15-52b1-4dc6-9473-69ae1347a2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:18.872692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d36e15-52b1-4dc6-9473-69ae1347a2f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:18.873780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d36e15-52b1-4dc6-9473-69ae1347a2f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:18.876378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d36e15-52b1-4dc6-9473-69ae1347a2f4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:18.916793 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 915/1839] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:18 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:27:18.917515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e952059-0065-497c-80ab-58a80ba242a9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:18.937396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e952059-0065-497c-80ab-58a80ba242a9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:18.944754 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a772e-c483-4336-8930-b20e1682be03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:18.945930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a772e-c483-4336-8930-b20e1682be03 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:18.946380 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a772e-c483-4336-8930-b20e1682be03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:18.947287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a772e-c483-4336-8930-b20e1682be03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:18.951537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a772e-c483-4336-8930-b20e1682be03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:18.953506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a772e-c483-4336-8930-b20e1682be03 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:18.970674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 925/1840] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:27:18.972594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d36e15-52b1-4dc6-9473-69ae1347a2f4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:18.989108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d36e15-52b1-4dc6-9473-69ae1347a2f4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:18.995185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e20ad-1b66-40cf-92b9-141e217a1838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:18.995864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e20ad-1b66-40cf-92b9-141e217a1838 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:18.996305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e20ad-1b66-40cf-92b9-141e217a1838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:18.996731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e20ad-1b66-40cf-92b9-141e217a1838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:18.997632 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e20ad-1b66-40cf-92b9-141e217a1838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:18.999458 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e20ad-1b66-40cf-92b9-141e217a1838 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:19.014112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a772e-c483-4336-8930-b20e1682be03 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:19.018402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2d8a772e-c483-4336-8930-b20e1682be03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:19.033541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 916/1841] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:19.048581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e20ad-1b66-40cf-92b9-141e217a1838 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:19.087719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e20ad-1b66-40cf-92b9-141e217a1838 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:19.102185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:19.108527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657bb12-e260-4ea2-a56e-ffe5c4edbf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:19.110300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657bb12-e260-4ea2-a56e-ffe5c4edbf6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:19.111357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657bb12-e260-4ea2-a56e-ffe5c4edbf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:19.112977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657bb12-e260-4ea2-a56e-ffe5c4edbf6b admin admin] PATH_INFO: `/v3/projects/690f6c2e43c74e83bcffcc49c17ffb32/users/c861281689c84440a67e081221c32803/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:19.143603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a772e-c483-4336-8930-b20e1682be03 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:19.160924 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657bb12-e260-4ea2-a56e-ffe5c4edbf6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=690f6c2e43c74e83bcffcc49c17ffb32, user_id=c861281689c84440a67e081221c32803, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:19.165204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79ccd5-e126-46d6-91a0-90ca6115ea5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:19.165905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ccd5-e126-46d6-91a0-90ca6115ea5f service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:19.166257 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ccd5-e126-46d6-91a0-90ca6115ea5f service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:19.166635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ccd5-e126-46d6-91a0-90ca6115ea5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:19.169918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657bb12-e260-4ea2-a56e-ffe5c4edbf6b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:19.176637 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8e79ccd5-e126-46d6-91a0-90ca6115ea5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:19.197920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 917/1842] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:19 2022] PUT /identity/v3/projects/690f6c2e43c74e83bcffcc49c17ffb32/users/c861281689c84440a67e081221c32803/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:19.202116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79ccd5-e126-46d6-91a0-90ca6115ea5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:19.203786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79ccd5-e126-46d6-91a0-90ca6115ea5f service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:19.209204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1247-5301-4316-85f9-16a72284ae6b None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:19.210226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1247-5301-4316-85f9-16a72284ae6b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:19.211998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1247-5301-4316-85f9-16a72284ae6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:19.221166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 926/1843] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:27:19.266277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-78cd1247-5301-4316-85f9-16a72284ae6b None None] MFA Rules not processed for user `c861281689c84440a67e081221c32803`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:19.271323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-78cd1247-5301-4316-85f9-16a72284ae6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:19.333827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:19.408187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 918/1844] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:27:19.413735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79ccd5-e126-46d6-91a0-90ca6115ea5f service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:19.428674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79ccd5-e126-46d6-91a0-90ca6115ea5f service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:19.622993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6368fe0-eabc-49a3-8746-be0540ca152a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:19.623811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6368fe0-eabc-49a3-8746-be0540ca152a admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:19.624100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6368fe0-eabc-49a3-8746-be0540ca152a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:19.624380 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6368fe0-eabc-49a3-8746-be0540ca152a admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:19.633053 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6368fe0-eabc-49a3-8746-be0540ca152a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:19.636795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6368fe0-eabc-49a3-8746-be0540ca152a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:19.715931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 927/1845] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:19 2022] POST /identity/v3/projects => generated 375 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:19.717663 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd1247-5301-4316-85f9-16a72284ae6b None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:19.736878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd1247-5301-4316-85f9-16a72284ae6b None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:19.868431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b93e1-e93a-4fb8-80ba-e45080ee8124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:19.869164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b93e1-e93a-4fb8-80ba-e45080ee8124 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:19.869451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b93e1-e93a-4fb8-80ba-e45080ee8124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:19.869725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b93e1-e93a-4fb8-80ba-e45080ee8124 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:19.870445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b93e1-e93a-4fb8-80ba-e45080ee8124 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:19.872775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b93e1-e93a-4fb8-80ba-e45080ee8124 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:19.915768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 919/1846] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:19 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:19.916536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6368fe0-eabc-49a3-8746-be0540ca152a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:19.932607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6368fe0-eabc-49a3-8746-be0540ca152a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:19.940687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c55dfd-c86a-4dc9-a772-4a6cbdd03a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:19.942582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c55dfd-c86a-4dc9-a772-4a6cbdd03a2f admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:19.943029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c55dfd-c86a-4dc9-a772-4a6cbdd03a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:19.943463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c55dfd-c86a-4dc9-a772-4a6cbdd03a2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:19.944622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c55dfd-c86a-4dc9-a772-4a6cbdd03a2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:19.962699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c55dfd-c86a-4dc9-a772-4a6cbdd03a2f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:19.984364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b93e1-e93a-4fb8-80ba-e45080ee8124 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:19.996001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 928/1847] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:19.999137 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b93e1-e93a-4fb8-80ba-e45080ee8124 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:20.005391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e4e5c-9b39-4e74-afc4-4e8095c4eddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:20.006879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e4e5c-9b39-4e74-afc4-4e8095c4eddd admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:20.025071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e4e5c-9b39-4e74-afc4-4e8095c4eddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:20.025389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e4e5c-9b39-4e74-afc4-4e8095c4eddd admin admin] PATH_INFO: `/v3/projects/570ba72ea79742378427da115b682bb7/users/8b2459bd3ffd462bab24e665d2636f29/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:20.062624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e4e5c-9b39-4e74-afc4-4e8095c4eddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=570ba72ea79742378427da115b682bb7, user_id=8b2459bd3ffd462bab24e665d2636f29, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:20.065626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e4e5c-9b39-4e74-afc4-4e8095c4eddd admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:20.093661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c55dfd-c86a-4dc9-a772-4a6cbdd03a2f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:20.094136 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 920/1848] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:19 2022] PUT /identity/v3/projects/570ba72ea79742378427da115b682bb7/users/8b2459bd3ffd462bab24e665d2636f29/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:20.106741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c55dfd-c86a-4dc9-a772-4a6cbdd03a2f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:20.226015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd7be6-36cd-438e-a8e3-29dac8c6bc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:20.228008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7be6-36cd-438e-a8e3-29dac8c6bc95 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:20.228549 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7be6-36cd-438e-a8e3-29dac8c6bc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:20.229056 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd7be6-36cd-438e-a8e3-29dac8c6bc95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:20.230746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd7be6-36cd-438e-a8e3-29dac8c6bc95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:20.232836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd7be6-36cd-438e-a8e3-29dac8c6bc95 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:20.250636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 929/1849] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:20.255083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e4e5c-9b39-4e74-afc4-4e8095c4eddd admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:20.273516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e4e5c-9b39-4e74-afc4-4e8095c4eddd admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:20.296662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3fbe1-41b4-4030-b9a6-66c4c2f3f56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:20.297583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3fbe1-41b4-4030-b9a6-66c4c2f3f56c admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:20.297860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3fbe1-41b4-4030-b9a6-66c4c2f3f56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:20.298130 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3fbe1-41b4-4030-b9a6-66c4c2f3f56c admin admin] PATH_INFO: `/v3/projects/570ba72ea79742378427da115b682bb7/users/8b2459bd3ffd462bab24e665d2636f29/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:20.300213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3fbe1-41b4-4030-b9a6-66c4c2f3f56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=570ba72ea79742378427da115b682bb7, user_id=8b2459bd3ffd462bab24e665d2636f29, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:20.303068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e3fbe1-41b4-4030-b9a6-66c4c2f3f56c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:20.327661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 921/1850] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:20 2022] PUT /identity/v3/projects/570ba72ea79742378427da115b682bb7/users/8b2459bd3ffd462bab24e665d2636f29/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:20.329556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db2ed5-ea46-42cc-b34f-5f22993f1338 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:20.329849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db2ed5-ea46-42cc-b34f-5f22993f1338 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:20.330155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db2ed5-ea46-42cc-b34f-5f22993f1338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:20.372129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-75db2ed5-ea46-42cc-b34f-5f22993f1338 None None] MFA Rules not processed for user `8b2459bd3ffd462bab24e665d2636f29`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:20.374119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-75db2ed5-ea46-42cc-b34f-5f22993f1338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:20.440959 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:20.468731 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 930/1851] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:20 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:20.479396 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3fbe1-41b4-4030-b9a6-66c4c2f3f56c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:20.481056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-38e3fbe1-41b4-4030-b9a6-66c4c2f3f56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:20.557079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3fbe1-41b4-4030-b9a6-66c4c2f3f56c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:20.560863 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f955bcd-2c72-44bb-8624-a596f17f2fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:20.561525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f955bcd-2c72-44bb-8624-a596f17f2fde service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:20.561809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f955bcd-2c72-44bb-8624-a596f17f2fde service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:20.562094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f955bcd-2c72-44bb-8624-a596f17f2fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:20.624675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:20.636962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f955bcd-2c72-44bb-8624-a596f17f2fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:20.638997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f955bcd-2c72-44bb-8624-a596f17f2fde service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:20.649686 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 922/1852] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:23.402818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db2ed5-ea46-42cc-b34f-5f22993f1338 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:23.505416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db2ed5-ea46-42cc-b34f-5f22993f1338 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:23.526927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:23.529360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:23.530441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:23.535576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] PATH_INFO: `/v3/users/c861281689c84440a67e081221c32803` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:23.537577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c861281689c84440a67e081221c32803)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:23.539783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:23.606589 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c861281689c84440a67e081221c32803'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:23.631547 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c861281689c84440a67e081221c32803'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:23.633202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c861281689c84440a67e081221c32803'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:23.643087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c861281689c84440a67e081221c32803'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:23.643922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c861281689c84440a67e081221c32803'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:23.670931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 931/1853] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:23 2022] DELETE /identity/v3/users/c861281689c84440a67e081221c32803 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:23.675889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665faf-f102-48ba-b58b-30ca7a779a87 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:23.676417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665faf-f102-48ba-b58b-30ca7a779a87 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:23.676672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665faf-f102-48ba-b58b-30ca7a779a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:23.697525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-60665faf-f102-48ba-b58b-30ca7a779a87 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:23.698941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-60665faf-f102-48ba-b58b-30ca7a779a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:23.815446 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 923/1854] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:27:23.830085 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:23.831591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:23.916909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8b9f7-de8b-47fb-b282-dfe118a8148a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:23.921201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a72281-3934-43cc-8775-c1ef9544e377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:23.922126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a72281-3934-43cc-8775-c1ef9544e377 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:23.922478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a72281-3934-43cc-8775-c1ef9544e377 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:23.922877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a72281-3934-43cc-8775-c1ef9544e377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:23.939206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a72281-3934-43cc-8775-c1ef9544e377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:23.942661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a72281-3934-43cc-8775-c1ef9544e377 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:23.956428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 932/1855] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:24.364171 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60665faf-f102-48ba-b58b-30ca7a779a87 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:24.365721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-60665faf-f102-48ba-b58b-30ca7a779a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:24.387193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60665faf-f102-48ba-b58b-30ca7a779a87 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:24.393377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:24.393377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:24.394086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:24.396620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] PATH_INFO: `/v3/projects/690f6c2e43c74e83bcffcc49c17ffb32` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:24.396620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=690f6c2e43c74e83bcffcc49c17ffb32)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:24.398596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:24.460499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690f6c2e43c74e83bcffcc49c17ffb32', 'request_id': 'req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:24.466027 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690f6c2e43c74e83bcffcc49c17ffb32', 'request_id': 'req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:24.476684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690f6c2e43c74e83bcffcc49c17ffb32', 'request_id': 'req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:24.492993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] The token cache is being invalidate because project 690f6c2e43c74e83bcffcc49c17ffb32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:24.493413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:24.494130 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:24.496708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 924/1856] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:24 2022] DELETE /identity/v3/projects/690f6c2e43c74e83bcffcc49c17ffb32 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:24.500668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a72281-3934-43cc-8775-c1ef9544e377 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:24.505804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:24.513901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a72281-3934-43cc-8775-c1ef9544e377 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:24.593547 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab04ae-2a99-431a-9080-4802180139c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:24.594479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab04ae-2a99-431a-9080-4802180139c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:24.594996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab04ae-2a99-431a-9080-4802180139c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:24.595306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab04ae-2a99-431a-9080-4802180139c8 admin admin] PATH_INFO: `/v3/users/8b2459bd3ffd462bab24e665d2636f29` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:24.596695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab04ae-2a99-431a-9080-4802180139c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2459bd3ffd462bab24e665d2636f29)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:24.599302 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab04ae-2a99-431a-9080-4802180139c8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:24.661385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6fab04ae-2a99-431a-9080-4802180139c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2459bd3ffd462bab24e665d2636f29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:24.676865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6fab04ae-2a99-431a-9080-4802180139c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2459bd3ffd462bab24e665d2636f29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:24.677951 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6fab04ae-2a99-431a-9080-4802180139c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2459bd3ffd462bab24e665d2636f29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:24.687867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6fab04ae-2a99-431a-9080-4802180139c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2459bd3ffd462bab24e665d2636f29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:24.688562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6fab04ae-2a99-431a-9080-4802180139c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2459bd3ffd462bab24e665d2636f29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:24.704886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 933/1857] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:24 2022] DELETE /identity/v3/users/8b2459bd3ffd462bab24e665d2636f29 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:25.062198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:25.063743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:25.079091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637880f7-7bdd-4d56-ad2f-3f24eaeb7c6b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:25.139859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:25.140655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:25.140933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:25.141204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] PATH_INFO: `/v3/projects/570ba72ea79742378427da115b682bb7` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:25.142406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=570ba72ea79742378427da115b682bb7)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:25.144393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:25.161993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:25.195077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '570ba72ea79742378427da115b682bb7', 'request_id': 'req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:25.201139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '570ba72ea79742378427da115b682bb7', 'request_id': 'req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:25.211920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '570ba72ea79742378427da115b682bb7', 'request_id': 'req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:25.224377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] The token cache is being invalidate because project 570ba72ea79742378427da115b682bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:25.224661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:25.225485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-6bd5188d-fc6d-4c2a-96b1-ff38bb034989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:25.228132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 925/1858] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:25 2022] DELETE /identity/v3/projects/570ba72ea79742378427da115b682bb7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:25.234469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38208d43-a7df-474a-88b1-b052af711ed4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:25.234794 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38208d43-a7df-474a-88b1-b052af711ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:25.235077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38208d43-a7df-474a-88b1-b052af711ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:25.265348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-38208d43-a7df-474a-88b1-b052af711ed4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:25.266721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-38208d43-a7df-474a-88b1-b052af711ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:25.351814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 934/1859] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:25.353886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472991d-0349-4311-883f-794b70752077 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:25.354182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472991d-0349-4311-883f-794b70752077 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:25.354465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472991d-0349-4311-883f-794b70752077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:25.383933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2472991d-0349-4311-883f-794b70752077 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:25.385018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2472991d-0349-4311-883f-794b70752077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:25.394123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 926/1860] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:25.399329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38208d43-a7df-474a-88b1-b052af711ed4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:25.410948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38208d43-a7df-474a-88b1-b052af711ed4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:25.415762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bfab1-ddba-4e80-9759-5eb0329ba20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:25.416437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bfab1-ddba-4e80-9759-5eb0329ba20a admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:25.416721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bfab1-ddba-4e80-9759-5eb0329ba20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:25.416998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bfab1-ddba-4e80-9759-5eb0329ba20a admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:25.417758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bfab1-ddba-4e80-9759-5eb0329ba20a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:25.418135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bfab1-ddba-4e80-9759-5eb0329ba20a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:25.419724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bfab1-ddba-4e80-9759-5eb0329ba20a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:25.433552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 935/1861] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:27:25.438310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472991d-0349-4311-883f-794b70752077 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:25.449346 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472991d-0349-4311-883f-794b70752077 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:25.457274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92e6a0-4198-4b78-a677-b4ea840e8dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:25.458667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92e6a0-4198-4b78-a677-b4ea840e8dac admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:25.459101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92e6a0-4198-4b78-a677-b4ea840e8dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:25.460903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92e6a0-4198-4b78-a677-b4ea840e8dac admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:25.460903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92e6a0-4198-4b78-a677-b4ea840e8dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:25.462159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92e6a0-4198-4b78-a677-b4ea840e8dac admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:25.497006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 927/1862] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:25 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:25.501287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bfab1-ddba-4e80-9759-5eb0329ba20a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:25.512017 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bfab1-ddba-4e80-9759-5eb0329ba20a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:25.517192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:25.585618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76fdf5-d41d-4304-99d3-799fdcdedfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:25.586326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76fdf5-d41d-4304-99d3-799fdcdedfc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:25.586652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76fdf5-d41d-4304-99d3-799fdcdedfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:25.586956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76fdf5-d41d-4304-99d3-799fdcdedfc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:25.587595 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a76fdf5-d41d-4304-99d3-799fdcdedfc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:25.589946 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a76fdf5-d41d-4304-99d3-799fdcdedfc1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:25.615215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 936/1863] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:25 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:25.616985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92e6a0-4198-4b78-a677-b4ea840e8dac admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:25.627668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92e6a0-4198-4b78-a677-b4ea840e8dac admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:25.632701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c4a09-6931-4457-b972-afff5ef47fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:25.633377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c4a09-6931-4457-b972-afff5ef47fef admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:25.633662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c4a09-6931-4457-b972-afff5ef47fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:25.633941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c4a09-6931-4457-b972-afff5ef47fef admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:25.634705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c4a09-6931-4457-b972-afff5ef47fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:25.636375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c4a09-6931-4457-b972-afff5ef47fef admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:25.651219 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 928/1864] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:25.652432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76fdf5-d41d-4304-99d3-799fdcdedfc1 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:25.664193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76fdf5-d41d-4304-99d3-799fdcdedfc1 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:25.669149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842f58a-317b-47bb-a2b8-4d9f9860238f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:25.669997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842f58a-317b-47bb-a2b8-4d9f9860238f admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:25.670565 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842f58a-317b-47bb-a2b8-4d9f9860238f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:25.674682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842f58a-317b-47bb-a2b8-4d9f9860238f admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:25.674682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842f58a-317b-47bb-a2b8-4d9f9860238f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:25.674682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842f58a-317b-47bb-a2b8-4d9f9860238f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:25.684891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 937/1865] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:25.689291 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c4a09-6931-4457-b972-afff5ef47fef admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:25.702411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c4a09-6931-4457-b972-afff5ef47fef admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:25.707552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e8457-855a-44be-8066-b7b0562ab8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:25.708445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e8457-855a-44be-8066-b7b0562ab8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:25.708729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e8457-855a-44be-8066-b7b0562ab8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:25.709081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e8457-855a-44be-8066-b7b0562ab8a4 admin admin] PATH_INFO: `/v3/projects/262e4e5ea40545049723e0007b952dc8/users/fc862db700c642bcbdb3c60ec72859a0/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:25.728680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e8457-855a-44be-8066-b7b0562ab8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=262e4e5ea40545049723e0007b952dc8, user_id=fc862db700c642bcbdb3c60ec72859a0, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:25.731557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e8457-855a-44be-8066-b7b0562ab8a4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:25.746267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 929/1866] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:25 2022] PUT /identity/v3/projects/262e4e5ea40545049723e0007b952dc8/users/fc862db700c642bcbdb3c60ec72859a0/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:25.756817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7631fe-97f1-43a0-8b6e-36435051aa06 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:25.757236 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7631fe-97f1-43a0-8b6e-36435051aa06 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:25.757519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7631fe-97f1-43a0-8b6e-36435051aa06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:25.786069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-eb7631fe-97f1-43a0-8b6e-36435051aa06 None None] MFA Rules not processed for user `fc862db700c642bcbdb3c60ec72859a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:25.787359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-eb7631fe-97f1-43a0-8b6e-36435051aa06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:25.856385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 938/1867] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:25.868322 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e8457-855a-44be-8066-b7b0562ab8a4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:25.870326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a69e8457-855a-44be-8066-b7b0562ab8a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:25.934394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e8457-855a-44be-8066-b7b0562ab8a4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:25.938628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41331384-ac1f-4c3d-ba46-f9cd9e32e21e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:25.939278 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41331384-ac1f-4c3d-ba46-f9cd9e32e21e service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:25.939557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41331384-ac1f-4c3d-ba46-f9cd9e32e21e service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:25.939835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41331384-ac1f-4c3d-ba46-f9cd9e32e21e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:25.948932 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41331384-ac1f-4c3d-ba46-f9cd9e32e21e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:25.950394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41331384-ac1f-4c3d-ba46-f9cd9e32e21e service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:25.962616 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 930/1868] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:25.987570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7631fe-97f1-43a0-8b6e-36435051aa06 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:26.066393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7631fe-97f1-43a0-8b6e-36435051aa06 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:26.071042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:26.071920 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:26.072201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:26.072471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] PATH_INFO: `/v3/users/fc862db700c642bcbdb3c60ec72859a0` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:26.074292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc862db700c642bcbdb3c60ec72859a0)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:26.076345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:26.102356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:26.137762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc862db700c642bcbdb3c60ec72859a0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:26.150858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc862db700c642bcbdb3c60ec72859a0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:26.151570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc862db700c642bcbdb3c60ec72859a0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:26.158353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc862db700c642bcbdb3c60ec72859a0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:26.159006 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc862db700c642bcbdb3c60ec72859a0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:26.175916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 939/1869] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:25 2022] DELETE /identity/v3/users/fc862db700c642bcbdb3c60ec72859a0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:26.180709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f8363-63ec-4e57-81ee-fc644d157df8 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:26.181870 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f8363-63ec-4e57-81ee-fc644d157df8 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:26.181870 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f8363-63ec-4e57-81ee-fc644d157df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:26.199367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-962f8363-63ec-4e57-81ee-fc644d157df8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:26.200572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-962f8363-63ec-4e57-81ee-fc644d157df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:26.243071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:26.264186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 931/1870] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:27:26.274460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:26.276004 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:26.338519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5950425-cf90-4008-90cf-6869311d0a4f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:26.342502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1bd21-765b-46bb-911c-1837b10e54cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:26.343184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1bd21-765b-46bb-911c-1837b10e54cf service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:26.343478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1bd21-765b-46bb-911c-1837b10e54cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:26.343740 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1bd21-765b-46bb-911c-1837b10e54cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:26.354109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1bd21-765b-46bb-911c-1837b10e54cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:26.355636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1bd21-765b-46bb-911c-1837b10e54cf service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:26.368532 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 940/1871] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:26.749487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f8363-63ec-4e57-81ee-fc644d157df8 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:26.752544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-962f8363-63ec-4e57-81ee-fc644d157df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:26.765465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f8363-63ec-4e57-81ee-fc644d157df8 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:26.776442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:26.778095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:26.778394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:26.778703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] PATH_INFO: `/v3/projects/262e4e5ea40545049723e0007b952dc8` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:26.779982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=262e4e5ea40545049723e0007b952dc8)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:26.782195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:26.841404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '262e4e5ea40545049723e0007b952dc8', 'request_id': 'req-2e8f7171-a7e1-47c9-801b-f671e24d96c0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:26.847298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '262e4e5ea40545049723e0007b952dc8', 'request_id': 'req-2e8f7171-a7e1-47c9-801b-f671e24d96c0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:26.859950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '262e4e5ea40545049723e0007b952dc8', 'request_id': 'req-2e8f7171-a7e1-47c9-801b-f671e24d96c0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:26.873393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] The token cache is being invalidate because project 262e4e5ea40545049723e0007b952dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:26.873705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:26.874363 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2e8f7171-a7e1-47c9-801b-f671e24d96c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:26.876215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 932/1872] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:26 2022] DELETE /identity/v3/projects/262e4e5ea40545049723e0007b952dc8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:26.880862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34e097-727f-4eb0-8048-b729becd2135 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:26.881384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34e097-727f-4eb0-8048-b729becd2135 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:26.881556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34e097-727f-4eb0-8048-b729becd2135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:26.905366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2a34e097-727f-4eb0-8048-b729becd2135 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:26.906719 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2a34e097-727f-4eb0-8048-b729becd2135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:26.991963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 941/1873] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:27:27.000813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6619cb6-b4bf-4a34-8a84-6c7797231206 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:27.001110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6619cb6-b4bf-4a34-8a84-6c7797231206 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:27.001394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6619cb6-b4bf-4a34-8a84-6c7797231206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:27.020988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-f6619cb6-b4bf-4a34-8a84-6c7797231206 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:27.022215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f6619cb6-b4bf-4a34-8a84-6c7797231206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:27.031012 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 933/1874] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:27.035384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34e097-727f-4eb0-8048-b729becd2135 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:27.045750 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34e097-727f-4eb0-8048-b729becd2135 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:27.050759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca061294-f7b8-4cf3-a2df-d5caec7ab974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:27.051498 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca061294-f7b8-4cf3-a2df-d5caec7ab974 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:27.051791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca061294-f7b8-4cf3-a2df-d5caec7ab974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:27.052071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca061294-f7b8-4cf3-a2df-d5caec7ab974 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:27.052803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca061294-f7b8-4cf3-a2df-d5caec7ab974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:27.053113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca061294-f7b8-4cf3-a2df-d5caec7ab974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:27.054850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca061294-f7b8-4cf3-a2df-d5caec7ab974 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:27.070603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 942/1875] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:27:27.071671 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6619cb6-b4bf-4a34-8a84-6c7797231206 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:27.083871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6619cb6-b4bf-4a34-8a84-6c7797231206 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:27.089113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df143a33-4c3c-4df6-9664-fb77cc130d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:27.089787 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df143a33-4c3c-4df6-9664-fb77cc130d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:27.090062 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df143a33-4c3c-4df6-9664-fb77cc130d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:27.090340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df143a33-4c3c-4df6-9664-fb77cc130d95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:27.091077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df143a33-4c3c-4df6-9664-fb77cc130d95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:27.093002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df143a33-4c3c-4df6-9664-fb77cc130d95 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:27.120576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 934/1876] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:27 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:27.125265 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca061294-f7b8-4cf3-a2df-d5caec7ab974 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:27.135878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca061294-f7b8-4cf3-a2df-d5caec7ab974 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:27.161482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:27.201000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7ddee-c6d0-444f-8845-02dc3ee3a97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:27.201817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7ddee-c6d0-444f-8845-02dc3ee3a97e admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:27.202110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7ddee-c6d0-444f-8845-02dc3ee3a97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:27.202393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7ddee-c6d0-444f-8845-02dc3ee3a97e admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:27.203131 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7ddee-c6d0-444f-8845-02dc3ee3a97e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:27.205233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7ddee-c6d0-444f-8845-02dc3ee3a97e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:27.227751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 943/1877] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:27 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:27.231782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df143a33-4c3c-4df6-9664-fb77cc130d95 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:27.242499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df143a33-4c3c-4df6-9664-fb77cc130d95 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:27.247467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394295c-e03f-404b-aa58-95366f3de90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:27.248242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394295c-e03f-404b-aa58-95366f3de90d admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:27.248596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394295c-e03f-404b-aa58-95366f3de90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:27.248946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394295c-e03f-404b-aa58-95366f3de90d admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:27.249755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394295c-e03f-404b-aa58-95366f3de90d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:27.251486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394295c-e03f-404b-aa58-95366f3de90d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:27.262775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 935/1878] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:27.266997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7ddee-c6d0-444f-8845-02dc3ee3a97e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:27.277986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7ddee-c6d0-444f-8845-02dc3ee3a97e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:27.283816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb3b61-71e9-4533-a270-740b3d4348e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:27.284563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3b61-71e9-4533-a270-740b3d4348e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:27.284866 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3b61-71e9-4533-a270-740b3d4348e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:27.285159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb3b61-71e9-4533-a270-740b3d4348e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:27.285895 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb3b61-71e9-4533-a270-740b3d4348e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:27.287623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb3b61-71e9-4533-a270-740b3d4348e9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:27.306198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 944/1879] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:27:27.307707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394295c-e03f-404b-aa58-95366f3de90d admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:27.318298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394295c-e03f-404b-aa58-95366f3de90d admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:27.323812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953da1d4-d801-4c9c-a492-69e9c05faf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:27.324712 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953da1d4-d801-4c9c-a492-69e9c05faf0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:27.324998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953da1d4-d801-4c9c-a492-69e9c05faf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:27.325290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953da1d4-d801-4c9c-a492-69e9c05faf0a admin admin] PATH_INFO: `/v3/projects/00d2022cc4be4779af1edf1520c439ae/users/dac7f01263374a2cbdd892a2513f5ecd/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:27.341913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953da1d4-d801-4c9c-a492-69e9c05faf0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=00d2022cc4be4779af1edf1520c439ae, user_id=dac7f01263374a2cbdd892a2513f5ecd, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:27.344689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953da1d4-d801-4c9c-a492-69e9c05faf0a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:27.358061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 936/1880] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:27 2022] PUT /identity/v3/projects/00d2022cc4be4779af1edf1520c439ae/users/dac7f01263374a2cbdd892a2513f5ecd/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:27.366059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5076389-4a92-4c89-b873-677df00ea2c4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:27.366059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5076389-4a92-4c89-b873-677df00ea2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:27.366277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5076389-4a92-4c89-b873-677df00ea2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:27.384775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c5076389-4a92-4c89-b873-677df00ea2c4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:27.385994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c5076389-4a92-4c89-b873-677df00ea2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:27.444256 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 945/1881] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:27.454310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953da1d4-d801-4c9c-a492-69e9c05faf0a admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:27.456076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-953da1d4-d801-4c9c-a492-69e9c05faf0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:27.518971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953da1d4-d801-4c9c-a492-69e9c05faf0a admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:27.523919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925063c-c0e6-48ca-ace8-d569e2c5b957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:27.524591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925063c-c0e6-48ca-ace8-d569e2c5b957 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:27.524866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925063c-c0e6-48ca-ace8-d569e2c5b957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:27.525149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925063c-c0e6-48ca-ace8-d569e2c5b957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:27.541159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925063c-c0e6-48ca-ace8-d569e2c5b957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:27.542861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925063c-c0e6-48ca-ace8-d569e2c5b957 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:27.553316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 937/1882] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:32.604016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5076389-4a92-4c89-b873-677df00ea2c4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:32.605325 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c5076389-4a92-4c89-b873-677df00ea2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:32.621690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5076389-4a92-4c89-b873-677df00ea2c4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:32.626625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:32.627721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:32.628010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:32.628289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] PATH_INFO: `/v3/users/0a9d160d254c4a1a96ca157f304b753e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:32.629734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a9d160d254c4a1a96ca157f304b753e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:32.631881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:32.723480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a9d160d254c4a1a96ca157f304b753e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:32.756470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a9d160d254c4a1a96ca157f304b753e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:32.758512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a9d160d254c4a1a96ca157f304b753e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:32.770522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a9d160d254c4a1a96ca157f304b753e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:32.774972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a9d160d254c4a1a96ca157f304b753e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:32.807639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 946/1883] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:32 2022] DELETE /identity/v3/users/0a9d160d254c4a1a96ca157f304b753e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:32.813928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977710ec-47ee-41b6-bb71-4d00f18d29d3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:32.814466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977710ec-47ee-41b6-bb71-4d00f18d29d3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:32.815156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977710ec-47ee-41b6-bb71-4d00f18d29d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:32.829856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:32.836713 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-977710ec-47ee-41b6-bb71-4d00f18d29d3 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:32.840792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-977710ec-47ee-41b6-bb71-4d00f18d29d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:32.925740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 938/1884] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:32.949054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:32.950547 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:32.973761 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:33.023541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d13c232-04bc-4944-a61a-c2916c4e013f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:33.027997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09d4d2-ed72-4986-ba79-6a9cf0060072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:33.028916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09d4d2-ed72-4986-ba79-6a9cf0060072 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:33.029385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09d4d2-ed72-4986-ba79-6a9cf0060072 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:33.030012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09d4d2-ed72-4986-ba79-6a9cf0060072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:33.044182 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09d4d2-ed72-4986-ba79-6a9cf0060072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:33.046667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09d4d2-ed72-4986-ba79-6a9cf0060072 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:33.060806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 947/1885] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:34.428157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977710ec-47ee-41b6-bb71-4d00f18d29d3 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:34.434677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-977710ec-47ee-41b6-bb71-4d00f18d29d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:34.472468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977710ec-47ee-41b6-bb71-4d00f18d29d3 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:34.479521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:34.480784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:34.481412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:34.481896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] PATH_INFO: `/v3/projects/1378bfa1199549baba2a955a8545d6a0` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:34.483937 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1378bfa1199549baba2a955a8545d6a0)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:34.486443 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:34.593683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1378bfa1199549baba2a955a8545d6a0', 'request_id': 'req-47858ba3-8216-4f17-88b6-6b5d8456b5d5'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:34.620768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1378bfa1199549baba2a955a8545d6a0', 'request_id': 'req-47858ba3-8216-4f17-88b6-6b5d8456b5d5'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:34.636815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1378bfa1199549baba2a955a8545d6a0', 'request_id': 'req-47858ba3-8216-4f17-88b6-6b5d8456b5d5'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:34.668558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] The token cache is being invalidate because project 1378bfa1199549baba2a955a8545d6a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:34.668856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:34.674063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:34.679127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 939/1886] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:34 2022] DELETE /identity/v3/projects/1378bfa1199549baba2a955a8545d6a0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:34.683410 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09d4d2-ed72-4986-ba79-6a9cf0060072 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:34.707599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09d4d2-ed72-4986-ba79-6a9cf0060072 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:34.869472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:34.871379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:34.871379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:34.871379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] PATH_INFO: `/v3/users/382b51af0b034254b58882ca15796025` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:34.872184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=382b51af0b034254b58882ca15796025)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:34.874154 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:34.953906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382b51af0b034254b58882ca15796025'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:34.968339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382b51af0b034254b58882ca15796025'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:34.969096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382b51af0b034254b58882ca15796025'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:34.977471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382b51af0b034254b58882ca15796025'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:34.978199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382b51af0b034254b58882ca15796025'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:34.997545 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 948/1887] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:34 2022] DELETE /identity/v3/users/382b51af0b034254b58882ca15796025 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:35.267504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:35.269112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:35.294051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47858ba3-8216-4f17-88b6-6b5d8456b5d5 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:35.344422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:35.346057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:35.399286 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b60e3-6a4e-4ba7-9791-5096aeff8a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:35.400917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b60e3-6a4e-4ba7-9791-5096aeff8a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:35.401456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b60e3-6a4e-4ba7-9791-5096aeff8a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:35.402188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b60e3-6a4e-4ba7-9791-5096aeff8a88 admin admin] PATH_INFO: `/v3/users/66a20306e93547969ff205e1da22b5f0` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:35.404498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b60e3-6a4e-4ba7-9791-5096aeff8a88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a20306e93547969ff205e1da22b5f0)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:35.406464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b60e3-6a4e-4ba7-9791-5096aeff8a88 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:35.474577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:35.499648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:35.522632 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-921b60e3-6a4e-4ba7-9791-5096aeff8a88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a20306e93547969ff205e1da22b5f0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:35.540661 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a9bb7-3711-4cef-93ba-c742155ae6b7 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:35.550262 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:35.551473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:35.554225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-921b60e3-6a4e-4ba7-9791-5096aeff8a88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a20306e93547969ff205e1da22b5f0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:35.554225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-921b60e3-6a4e-4ba7-9791-5096aeff8a88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a20306e93547969ff205e1da22b5f0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:35.559032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:35.559596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] PATH_INFO: `/v3/projects/bf9eed1ff39e42869fc520564c6260c5` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:35.563774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf9eed1ff39e42869fc520564c6260c5)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:35.567712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:35.568899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-921b60e3-6a4e-4ba7-9791-5096aeff8a88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a20306e93547969ff205e1da22b5f0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:35.569990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-921b60e3-6a4e-4ba7-9791-5096aeff8a88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a20306e93547969ff205e1da22b5f0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:35.607428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 940/1888] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:35 2022] DELETE /identity/v3/users/66a20306e93547969ff205e1da22b5f0 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:35.611489 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4dad9-7ce9-41f2-a97c-89367a7177f6 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:35.612814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4dad9-7ce9-41f2-a97c-89367a7177f6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:35.614972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d4dad9-7ce9-41f2-a97c-89367a7177f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:35.679118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf9eed1ff39e42869fc520564c6260c5', 'request_id': 'req-5771461c-a9e9-4def-800c-b1758afddaa1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:35.685646 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf9eed1ff39e42869fc520564c6260c5', 'request_id': 'req-5771461c-a9e9-4def-800c-b1758afddaa1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:35.695364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-58d4dad9-7ce9-41f2-a97c-89367a7177f6 None None] MFA Rules not processed for user `dac7f01263374a2cbdd892a2513f5ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:35.696424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-58d4dad9-7ce9-41f2-a97c-89367a7177f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:35.715387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf9eed1ff39e42869fc520564c6260c5', 'request_id': 'req-5771461c-a9e9-4def-800c-b1758afddaa1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:35.735138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] The token cache is being invalidate because project bf9eed1ff39e42869fc520564c6260c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:35.735596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:35.736472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-5771461c-a9e9-4def-800c-b1758afddaa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:35.738234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 949/1889] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:35 2022] DELETE /identity/v3/projects/bf9eed1ff39e42869fc520564c6260c5 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:35.740882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f7b6b-0768-4a49-82ed-91e6235ca3a4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:35.741363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f7b6b-0768-4a49-82ed-91e6235ca3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:35.742504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f7b6b-0768-4a49-82ed-91e6235ca3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:35.769321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-610f7b6b-0768-4a49-82ed-91e6235ca3a4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:35.771407 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-610f7b6b-0768-4a49-82ed-91e6235ca3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:35.807674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 941/1890] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:35.809118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311463-fa13-4173-9a15-8dfac4fd0075 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:35.809477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311463-fa13-4173-9a15-8dfac4fd0075 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:35.809769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311463-fa13-4173-9a15-8dfac4fd0075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:35.844021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-6f311463-fa13-4173-9a15-8dfac4fd0075 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:35.845584 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6f311463-fa13-4173-9a15-8dfac4fd0075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:35.913876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 950/1891] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:35.917081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f7b6b-0768-4a49-82ed-91e6235ca3a4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:35.926683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-610f7b6b-0768-4a49-82ed-91e6235ca3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:35.957745 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 942/1892] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:35.960030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a0301-e5f8-4dc0-933e-8e4b46bda189 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:35.960030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a0301-e5f8-4dc0-933e-8e4b46bda189 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:35.960260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a0301-e5f8-4dc0-933e-8e4b46bda189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:35.972600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f7b6b-0768-4a49-82ed-91e6235ca3a4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:35.986635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-bc9a0301-e5f8-4dc0-933e-8e4b46bda189 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:35.988109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-bc9a0301-e5f8-4dc0-933e-8e4b46bda189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:35.988666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96def2df-0953-472c-add4-9658885b23ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:35.990756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96def2df-0953-472c-add4-9658885b23ca admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:35.991805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96def2df-0953-472c-add4-9658885b23ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:35.992226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96def2df-0953-472c-add4-9658885b23ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:35.993097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96def2df-0953-472c-add4-9658885b23ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:35.997559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96def2df-0953-472c-add4-9658885b23ca admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.001730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 943/1893] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:27:36.005478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a0301-e5f8-4dc0-933e-8e4b46bda189 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.009343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-bc9a0301-e5f8-4dc0-933e-8e4b46bda189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:36.063715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 951/1894] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:35 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:36.064688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96def2df-0953-472c-add4-9658885b23ca admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.073289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:36.115595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a0301-e5f8-4dc0-933e-8e4b46bda189 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.142522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:36.169607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96def2df-0953-472c-add4-9658885b23ca admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.176901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d1e15-5d32-4ddf-99bc-943ffc411a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.177856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d1e15-5d32-4ddf-99bc-943ffc411a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.178184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d1e15-5d32-4ddf-99bc-943ffc411a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.179648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d1e15-5d32-4ddf-99bc-943ffc411a8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.180689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d1e15-5d32-4ddf-99bc-943ffc411a8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:36.181078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d1e15-5d32-4ddf-99bc-943ffc411a8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.183249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12d1e15-5d32-4ddf-99bc-943ffc411a8e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.205555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 952/1895] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:27:36.205555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d1e15-5d32-4ddf-99bc-943ffc411a8e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.217672 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d1e15-5d32-4ddf-99bc-943ffc411a8e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.224889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984d0bb-5dd8-49ee-9a5c-9756c77e0ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.226581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984d0bb-5dd8-49ee-9a5c-9756c77e0ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.226869 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984d0bb-5dd8-49ee-9a5c-9756c77e0ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.227150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984d0bb-5dd8-49ee-9a5c-9756c77e0ec5 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.228426 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984d0bb-5dd8-49ee-9a5c-9756c77e0ec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.231128 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984d0bb-5dd8-49ee-9a5c-9756c77e0ec5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.233145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0219348-293b-472d-ac8f-76c5b5545404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.234423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0219348-293b-472d-ac8f-76c5b5545404 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.234858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0219348-293b-472d-ac8f-76c5b5545404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.235296 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0219348-293b-472d-ac8f-76c5b5545404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.248138 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0219348-293b-472d-ac8f-76c5b5545404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.251042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0219348-293b-472d-ac8f-76c5b5545404 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.263543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 953/1896] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:36 2022] POST /identity/v3/users => generated 298 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:36.264581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984d0bb-5dd8-49ee-9a5c-9756c77e0ec5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.273173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 944/1897] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:36.274470 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0219348-293b-472d-ac8f-76c5b5545404 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.277378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984d0bb-5dd8-49ee-9a5c-9756c77e0ec5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.287310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c36ca-e5ad-4014-abbb-8f4ba13ce0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.289992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c36ca-e5ad-4014-abbb-8f4ba13ce0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.290402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c36ca-e5ad-4014-abbb-8f4ba13ce0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.290811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c36ca-e5ad-4014-abbb-8f4ba13ce0f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.291803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c36ca-e5ad-4014-abbb-8f4ba13ce0f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.295822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c36ca-e5ad-4014-abbb-8f4ba13ce0f3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.297198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0219348-293b-472d-ac8f-76c5b5545404 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.309893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a9395-86de-4ec2-9177-408f42ca7d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.310900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a9395-86de-4ec2-9177-408f42ca7d81 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.311485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a9395-86de-4ec2-9177-408f42ca7d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.312118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a9395-86de-4ec2-9177-408f42ca7d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.312531 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a9395-86de-4ec2-9177-408f42ca7d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.314220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a9395-86de-4ec2-9177-408f42ca7d81 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.330404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 945/1898] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:36.336554 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a9395-86de-4ec2-9177-408f42ca7d81 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.348403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 954/1899] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:36 2022] POST /identity/v3/projects => generated 367 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:36.349603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c36ca-e5ad-4014-abbb-8f4ba13ce0f3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.364967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a9395-86de-4ec2-9177-408f42ca7d81 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.475926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87eb2d6-b2a0-4e91-ade7-726991f0beee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.478283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eb2d6-b2a0-4e91-ade7-726991f0beee admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.479436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eb2d6-b2a0-4e91-ade7-726991f0beee admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.480128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eb2d6-b2a0-4e91-ade7-726991f0beee admin admin] PATH_INFO: `/v3/projects/433937ccb338451d8b27f5be7c1d78fa/users/b876d4f582d0478093e0bb22fbf07ecf/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.499129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c36ca-e5ad-4014-abbb-8f4ba13ce0f3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.513534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87eb2d6-b2a0-4e91-ade7-726991f0beee admin admin] RBAC: Authorizing `identity:create_grant(project_id=433937ccb338451d8b27f5be7c1d78fa, user_id=b876d4f582d0478093e0bb22fbf07ecf, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.514012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c657196-0b40-4d5f-ad82-f248cd562a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.515503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c657196-0b40-4d5f-ad82-f248cd562a2c admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.515804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c657196-0b40-4d5f-ad82-f248cd562a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.516093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c657196-0b40-4d5f-ad82-f248cd562a2c admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.516992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c657196-0b40-4d5f-ad82-f248cd562a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.518306 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87eb2d6-b2a0-4e91-ade7-726991f0beee admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.518995 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c657196-0b40-4d5f-ad82-f248cd562a2c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.536754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 946/1900] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:36 2022] PUT /identity/v3/projects/433937ccb338451d8b27f5be7c1d78fa/users/b876d4f582d0478093e0bb22fbf07ecf/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:36.541178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87eb2d6-b2a0-4e91-ade7-726991f0beee admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.574613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 955/1901] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:36 2022] POST /identity/v3/users => generated 284 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:36.575703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c657196-0b40-4d5f-ad82-f248cd562a2c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.589952 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c657196-0b40-4d5f-ad82-f248cd562a2c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.670752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87eb2d6-b2a0-4e91-ade7-726991f0beee admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.677437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15ae63-99de-40c3-a2eb-4478be2abe45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.679921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15ae63-99de-40c3-a2eb-4478be2abe45 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.680488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15ae63-99de-40c3-a2eb-4478be2abe45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.680962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15ae63-99de-40c3-a2eb-4478be2abe45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.682032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15ae63-99de-40c3-a2eb-4478be2abe45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.685017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15ae63-99de-40c3-a2eb-4478be2abe45 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.688652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af9e21-0a63-4191-9c69-e4a90a7121dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.689554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af9e21-0a63-4191-9c69-e4a90a7121dc admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.689962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af9e21-0a63-4191-9c69-e4a90a7121dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.690199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af9e21-0a63-4191-9c69-e4a90a7121dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.691033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af9e21-0a63-4191-9c69-e4a90a7121dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.693519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af9e21-0a63-4191-9c69-e4a90a7121dc admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.697582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:36.705726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 947/1902] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:36.710981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15ae63-99de-40c3-a2eb-4478be2abe45 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.711739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 956/1903] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:36.715002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af9e21-0a63-4191-9c69-e4a90a7121dc admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.732912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15ae63-99de-40c3-a2eb-4478be2abe45 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.736390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af9e21-0a63-4191-9c69-e4a90a7121dc admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.740170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5b04d-d679-4b84-8587-9ca48a786f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.741334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5b04d-d679-4b84-8587-9ca48a786f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.742653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5b04d-d679-4b84-8587-9ca48a786f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.742943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5b04d-d679-4b84-8587-9ca48a786f50 admin admin] PATH_INFO: `/v3/projects/433937ccb338451d8b27f5be7c1d78fa/users/b876d4f582d0478093e0bb22fbf07ecf/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.743160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aafa0-9abd-4a52-b2a7-e3b42cfdcb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.744098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aafa0-9abd-4a52-b2a7-e3b42cfdcb96 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.744582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aafa0-9abd-4a52-b2a7-e3b42cfdcb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.745088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23aafa0-9abd-4a52-b2a7-e3b42cfdcb96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.745973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5b04d-d679-4b84-8587-9ca48a786f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=433937ccb338451d8b27f5be7c1d78fa, user_id=b876d4f582d0478093e0bb22fbf07ecf, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.747307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23aafa0-9abd-4a52-b2a7-e3b42cfdcb96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.749411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5b04d-d679-4b84-8587-9ca48a786f50 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.750097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23aafa0-9abd-4a52-b2a7-e3b42cfdcb96 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.764218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 957/1904] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:36.767494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 948/1905] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:36 2022] PUT /identity/v3/projects/433937ccb338451d8b27f5be7c1d78fa/users/b876d4f582d0478093e0bb22fbf07ecf/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:36.767770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aafa0-9abd-4a52-b2a7-e3b42cfdcb96 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:36.786016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23aafa0-9abd-4a52-b2a7-e3b42cfdcb96 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:36.804430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:36.863160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eaed8-dc81-46a5-9dc8-5b8a86d2cde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:36.864071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eaed8-dc81-46a5-9dc8-5b8a86d2cde0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.864355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eaed8-dc81-46a5-9dc8-5b8a86d2cde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.864628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eaed8-dc81-46a5-9dc8-5b8a86d2cde0 admin admin] PATH_INFO: `/v3/projects/f466b053940e457b96a8e4a40f9a3879/users/25eed8a95a5d490b963e0c08f1460f15/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:36.883286 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792eaed8-dc81-46a5-9dc8-5b8a86d2cde0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f466b053940e457b96a8e4a40f9a3879, user_id=25eed8a95a5d490b963e0c08f1460f15, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:36.887532 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792eaed8-dc81-46a5-9dc8-5b8a86d2cde0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:36.916387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 958/1906] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:36 2022] PUT /identity/v3/projects/f466b053940e457b96a8e4a40f9a3879/users/25eed8a95a5d490b963e0c08f1460f15/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:36.928640 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73830910-f6b6-4bf3-b320-f05756c40cb2 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:36.929710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73830910-f6b6-4bf3-b320-f05756c40cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:36.930152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73830910-f6b6-4bf3-b320-f05756c40cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:37.012696 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-73830910-f6b6-4bf3-b320-f05756c40cb2 None None] MFA Rules not processed for user `25eed8a95a5d490b963e0c08f1460f15`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:37.015077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-73830910-f6b6-4bf3-b320-f05756c40cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:37.154002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eaed8-dc81-46a5-9dc8-5b8a86d2cde0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:37.156223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 949/1907] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:37.269081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eaed8-dc81-46a5-9dc8-5b8a86d2cde0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:37.285462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3ac16-06d7-44c5-b663-f6d109e98eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:37.286445 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3ac16-06d7-44c5-b663-f6d109e98eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:37.286896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3ac16-06d7-44c5-b663-f6d109e98eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:37.287316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc3ac16-06d7-44c5-b663-f6d109e98eb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:37.289025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3ac16-06d7-44c5-b663-f6d109e98eb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:37.295572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc3ac16-06d7-44c5-b663-f6d109e98eb4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:37.358210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73830910-f6b6-4bf3-b320-f05756c40cb2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:37.360223 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 959/1908] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:37 2022] POST /identity/v3/projects => generated 369 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:37.371140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73830910-f6b6-4bf3-b320-f05756c40cb2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:37.442371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb841bc7-f53b-4a69-8d5e-3435ea4b4d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:37.443000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb841bc7-f53b-4a69-8d5e-3435ea4b4d3f admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:37.443278 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb841bc7-f53b-4a69-8d5e-3435ea4b4d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:37.443705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb841bc7-f53b-4a69-8d5e-3435ea4b4d3f admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:37.445032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb841bc7-f53b-4a69-8d5e-3435ea4b4d3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:37.447386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb841bc7-f53b-4a69-8d5e-3435ea4b4d3f admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:37.479096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3ac16-06d7-44c5-b663-f6d109e98eb4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:37.492744 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 950/1909] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:37 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:27:37.497249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc3ac16-06d7-44c5-b663-f6d109e98eb4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:37.505052 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327b0f-6947-4150-aab3-2f3468e46fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:37.507177 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327b0f-6947-4150-aab3-2f3468e46fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:37.507575 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327b0f-6947-4150-aab3-2f3468e46fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:37.507956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8327b0f-6947-4150-aab3-2f3468e46fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:37.508891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8327b0f-6947-4150-aab3-2f3468e46fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:37.510757 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8327b0f-6947-4150-aab3-2f3468e46fe0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:37.525353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 960/1910] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:37.530415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb841bc7-f53b-4a69-8d5e-3435ea4b4d3f admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:37.542334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb841bc7-f53b-4a69-8d5e-3435ea4b4d3f admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:37.547594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4af825-ad28-4c3a-bbe3-958efcf66823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:37.548491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4af825-ad28-4c3a-bbe3-958efcf66823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:37.548768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4af825-ad28-4c3a-bbe3-958efcf66823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:37.549086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4af825-ad28-4c3a-bbe3-958efcf66823 admin admin] PATH_INFO: `/v3/projects/f2521b0a2aa040389b39f5a7e88734fb/users/4594ada515ef4fad9d28e4f3ee220843/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:37.570446 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4af825-ad28-4c3a-bbe3-958efcf66823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2521b0a2aa040389b39f5a7e88734fb, user_id=4594ada515ef4fad9d28e4f3ee220843, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:37.573723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4af825-ad28-4c3a-bbe3-958efcf66823 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:37.596808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327b0f-6947-4150-aab3-2f3468e46fe0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:37.597244 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 951/1911] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:37 2022] PUT /identity/v3/projects/f2521b0a2aa040389b39f5a7e88734fb/users/4594ada515ef4fad9d28e4f3ee220843/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:37.607792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8327b0f-6947-4150-aab3-2f3468e46fe0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:37.712762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3dd933-7df4-4c33-a168-69cb56bf946e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:37.713566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dd933-7df4-4c33-a168-69cb56bf946e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:37.713783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dd933-7df4-4c33-a168-69cb56bf946e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:37.714067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3dd933-7df4-4c33-a168-69cb56bf946e admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:37.714832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3dd933-7df4-4c33-a168-69cb56bf946e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:37.717979 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3dd933-7df4-4c33-a168-69cb56bf946e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:37.746843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4af825-ad28-4c3a-bbe3-958efcf66823 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:37.748066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 961/1912] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:37.763618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4af825-ad28-4c3a-bbe3-958efcf66823 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:37.769785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e9fa9-efcc-47e0-8ea1-8be614e06172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:37.770852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e9fa9-efcc-47e0-8ea1-8be614e06172 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:37.771135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e9fa9-efcc-47e0-8ea1-8be614e06172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:37.771411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e9fa9-efcc-47e0-8ea1-8be614e06172 admin admin] PATH_INFO: `/v3/projects/f2521b0a2aa040389b39f5a7e88734fb/users/4594ada515ef4fad9d28e4f3ee220843/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:37.774006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e9fa9-efcc-47e0-8ea1-8be614e06172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2521b0a2aa040389b39f5a7e88734fb, user_id=4594ada515ef4fad9d28e4f3ee220843, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:37.777166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e9fa9-efcc-47e0-8ea1-8be614e06172 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:37.799549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 952/1913] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:37 2022] PUT /identity/v3/projects/f2521b0a2aa040389b39f5a7e88734fb/users/4594ada515ef4fad9d28e4f3ee220843/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:37.805710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f401-f733-4004-81e2-201e0ed31e7b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:37.806032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f401-f733-4004-81e2-201e0ed31e7b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:37.806272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93f401-f733-4004-81e2-201e0ed31e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:37.828084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:37.840597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0e93f401-f733-4004-81e2-201e0ed31e7b None None] MFA Rules not processed for user `4594ada515ef4fad9d28e4f3ee220843`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:37.841831 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0e93f401-f733-4004-81e2-201e0ed31e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:37.949789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e9fa9-efcc-47e0-8ea1-8be614e06172 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:37.949941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 962/1914] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:37 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:37.951547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-913e9fa9-efcc-47e0-8ea1-8be614e06172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:37.962691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93f401-f733-4004-81e2-201e0ed31e7b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:37.965384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0e93f401-f733-4004-81e2-201e0ed31e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:37.988964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:38.104124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e9fa9-efcc-47e0-8ea1-8be614e06172 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:38.116566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:38.117435 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:38.117710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:38.117981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] PATH_INFO: `/v3/projects/af6882ed2e30401683064e262659ef63` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:38.126133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] RBAC: Authorizing `identity:delete_project(project_id=af6882ed2e30401683064e262659ef63)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:38.129601 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:38.145698 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93f401-f733-4004-81e2-201e0ed31e7b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:38.158005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a482fe1a-11e3-4d0a-bc55-fb7e991f7b2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:38.161886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482fe1a-11e3-4d0a-bc55-fb7e991f7b2a service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:38.161886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482fe1a-11e3-4d0a-bc55-fb7e991f7b2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:38.161886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a482fe1a-11e3-4d0a-bc55-fb7e991f7b2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:38.192813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482fe1a-11e3-4d0a-bc55-fb7e991f7b2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:38.198828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a482fe1a-11e3-4d0a-bc55-fb7e991f7b2a service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:38.215484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 963/1915] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:37 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:38.255327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af6882ed2e30401683064e262659ef63', 'request_id': 'req-72edd40e-77f0-4875-b494-e7c1bb84b74b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:38.264909 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af6882ed2e30401683064e262659ef63', 'request_id': 'req-72edd40e-77f0-4875-b494-e7c1bb84b74b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:38.276411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af6882ed2e30401683064e262659ef63', 'request_id': 'req-72edd40e-77f0-4875-b494-e7c1bb84b74b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:38.293602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] The token cache is being invalidate because project af6882ed2e30401683064e262659ef63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:38.293979 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:38.294880 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:38.303729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 953/1916] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:37 2022] DELETE /identity/v3/projects/af6882ed2e30401683064e262659ef63 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:38.304443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf70de-b752-4124-9621-3e151dd990a3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:38.306065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf70de-b752-4124-9621-3e151dd990a3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:38.306424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf70de-b752-4124-9621-3e151dd990a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:38.318025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:38.319559 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:38.334004 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-cbbf70de-b752-4124-9621-3e151dd990a3 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:38.336051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cbbf70de-b752-4124-9621-3e151dd990a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:38.424719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72edd40e-77f0-4875-b494-e7c1bb84b74b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:38.430388 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e94824-3659-45c6-92dc-9d10198723a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:38.431318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e94824-3659-45c6-92dc-9d10198723a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:38.431763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e94824-3659-45c6-92dc-9d10198723a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:38.432182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e94824-3659-45c6-92dc-9d10198723a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:38.434406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d3e94824-3659-45c6-92dc-9d10198723a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:38.468800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 964/1917] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:38.472020 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52feb86-0ace-4b34-a76f-59632b2bd3ee None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:38.472917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52feb86-0ace-4b34-a76f-59632b2bd3ee None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:38.474517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52feb86-0ace-4b34-a76f-59632b2bd3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:38.509820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e52feb86-0ace-4b34-a76f-59632b2bd3ee None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:38.511770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e52feb86-0ace-4b34-a76f-59632b2bd3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:38.527276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 965/1918] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:38.531138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52feb86-0ace-4b34-a76f-59632b2bd3ee None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:38.546795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52feb86-0ace-4b34-a76f-59632b2bd3ee None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:38.554983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f238b5f-8701-4f6c-92ef-3733304cd1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:38.556251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f238b5f-8701-4f6c-92ef-3733304cd1bc admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:38.556808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f238b5f-8701-4f6c-92ef-3733304cd1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:38.557584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f238b5f-8701-4f6c-92ef-3733304cd1bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:38.559828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f238b5f-8701-4f6c-92ef-3733304cd1bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:38.560567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f238b5f-8701-4f6c-92ef-3733304cd1bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:38.562795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f238b5f-8701-4f6c-92ef-3733304cd1bc admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:38.582225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 966/1919] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:27:38.587187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f238b5f-8701-4f6c-92ef-3733304cd1bc admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:38.594371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e94824-3659-45c6-92dc-9d10198723a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:38.597314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e94824-3659-45c6-92dc-9d10198723a6 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:38.603276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f238b5f-8701-4f6c-92ef-3733304cd1bc admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:38.612690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af86f5d-a68d-4aaa-9eaa-a2d017a4d073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:38.614031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af86f5d-a68d-4aaa-9eaa-a2d017a4d073 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:38.614683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af86f5d-a68d-4aaa-9eaa-a2d017a4d073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:38.628048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af86f5d-a68d-4aaa-9eaa-a2d017a4d073 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:38.630399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af86f5d-a68d-4aaa-9eaa-a2d017a4d073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:38.634770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af86f5d-a68d-4aaa-9eaa-a2d017a4d073 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:38.647123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 954/1920] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:38.698781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:38.712729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 967/1921] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:38 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:38.716549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e94824-3659-45c6-92dc-9d10198723a6 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:38.743243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e94824-3659-45c6-92dc-9d10198723a6 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:38.785329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:38.864670 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5447-334c-4a4e-8645-e770e7cae0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:38.868433 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5447-334c-4a4e-8645-e770e7cae0ca admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:38.868995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5447-334c-4a4e-8645-e770e7cae0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:38.869490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b5447-334c-4a4e-8645-e770e7cae0ca admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:38.870406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b5447-334c-4a4e-8645-e770e7cae0ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:38.872632 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2b5447-334c-4a4e-8645-e770e7cae0ca admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:38.920926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 955/1922] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:38 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:38.921270 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af86f5d-a68d-4aaa-9eaa-a2d017a4d073 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:38.942612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af86f5d-a68d-4aaa-9eaa-a2d017a4d073 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:38.956408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1767e8-8047-482a-891c-03c45b59a444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:38.960527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1767e8-8047-482a-891c-03c45b59a444 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:38.961065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1767e8-8047-482a-891c-03c45b59a444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:38.962466 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1767e8-8047-482a-891c-03c45b59a444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:38.964616 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1767e8-8047-482a-891c-03c45b59a444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:38.970214 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1767e8-8047-482a-891c-03c45b59a444 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:38.999953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 968/1923] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:39.005734 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5447-334c-4a4e-8645-e770e7cae0ca admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:39.033184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b5447-334c-4a4e-8645-e770e7cae0ca admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:39.047116 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff95940-8e79-4d7a-9e9f-2331425faf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:39.047809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff95940-8e79-4d7a-9e9f-2331425faf3b admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:39.048102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff95940-8e79-4d7a-9e9f-2331425faf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:39.048371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff95940-8e79-4d7a-9e9f-2331425faf3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:39.049117 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff95940-8e79-4d7a-9e9f-2331425faf3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:39.050847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff95940-8e79-4d7a-9e9f-2331425faf3b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:39.094486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 956/1924] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:39.095748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1767e8-8047-482a-891c-03c45b59a444 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:39.113329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1767e8-8047-482a-891c-03c45b59a444 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:39.124806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d343153-aaa2-4db2-a5ed-a2b2d85d1601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:39.128899 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d343153-aaa2-4db2-a5ed-a2b2d85d1601 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:39.129346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d343153-aaa2-4db2-a5ed-a2b2d85d1601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:39.129935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d343153-aaa2-4db2-a5ed-a2b2d85d1601 admin admin] PATH_INFO: `/v3/projects/5e4965cab58d4123b2a1e4fbc7f2c85d/users/41849d511bc14f65b8339629667a9480/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:39.184003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d343153-aaa2-4db2-a5ed-a2b2d85d1601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4965cab58d4123b2a1e4fbc7f2c85d, user_id=41849d511bc14f65b8339629667a9480, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:39.204691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d343153-aaa2-4db2-a5ed-a2b2d85d1601 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:39.232850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 969/1925] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:39 2022] PUT /identity/v3/projects/5e4965cab58d4123b2a1e4fbc7f2c85d/users/41849d511bc14f65b8339629667a9480/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:39.246964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142b2b0-1a5d-4d1c-aa72-106f35557d55 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:39.247257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142b2b0-1a5d-4d1c-aa72-106f35557d55 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:39.247550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142b2b0-1a5d-4d1c-aa72-106f35557d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:39.282149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0142b2b0-1a5d-4d1c-aa72-106f35557d55 None None] MFA Rules not processed for user `41849d511bc14f65b8339629667a9480`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:39.283795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0142b2b0-1a5d-4d1c-aa72-106f35557d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:39.473378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 957/1926] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:39.483261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d343153-aaa2-4db2-a5ed-a2b2d85d1601 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:39.628296 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d343153-aaa2-4db2-a5ed-a2b2d85d1601 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:39.648911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9daebb8-bc89-474b-943e-33139adb756d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:39.659865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daebb8-bc89-474b-943e-33139adb756d service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:39.662007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daebb8-bc89-474b-943e-33139adb756d service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:39.662467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9daebb8-bc89-474b-943e-33139adb756d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:39.695183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9daebb8-bc89-474b-943e-33139adb756d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:39.696729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9daebb8-bc89-474b-943e-33139adb756d service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:39.709280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 970/1927] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:43.587315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142b2b0-1a5d-4d1c-aa72-106f35557d55 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:43.590316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0142b2b0-1a5d-4d1c-aa72-106f35557d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:43.776364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142b2b0-1a5d-4d1c-aa72-106f35557d55 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:43.786312 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:43.787570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:43.788012 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:43.788661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] PATH_INFO: `/v3/users/25eed8a95a5d490b963e0c08f1460f15` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:43.790141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25eed8a95a5d490b963e0c08f1460f15)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:43.793658 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:43.830960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:43.892282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25eed8a95a5d490b963e0c08f1460f15'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:43.908775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25eed8a95a5d490b963e0c08f1460f15'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:43.910113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25eed8a95a5d490b963e0c08f1460f15'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:43.922177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25eed8a95a5d490b963e0c08f1460f15'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:43.923509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25eed8a95a5d490b963e0c08f1460f15'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:43.950282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 958/1928] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:43 2022] DELETE /identity/v3/users/25eed8a95a5d490b963e0c08f1460f15 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:43.957397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd57780-f0fb-436c-8040-65e73c8846ec None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:43.958415 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd57780-f0fb-436c-8040-65e73c8846ec None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:43.958836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd57780-f0fb-436c-8040-65e73c8846ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:43.983021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:43.984480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:43.988938 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6dd57780-f0fb-436c-8040-65e73c8846ec None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:43.991925 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6dd57780-f0fb-436c-8040-65e73c8846ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:44.090287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 971/1929] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:44.109284 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a2b36-4470-4261-a260-cc4af33f9e85 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:44.109463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd57780-f0fb-436c-8040-65e73c8846ec None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:44.110779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6dd57780-f0fb-436c-8040-65e73c8846ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:44.117139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af6715-f409-4b70-802f-38243f326d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:44.119698 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af6715-f409-4b70-802f-38243f326d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:44.120321 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af6715-f409-4b70-802f-38243f326d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:44.126135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af6715-f409-4b70-802f-38243f326d57 admin admin] PATH_INFO: `/v3/users/41849d511bc14f65b8339629667a9480` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:44.126135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af6715-f409-4b70-802f-38243f326d57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41849d511bc14f65b8339629667a9480)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:44.126135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91af6715-f409-4b70-802f-38243f326d57 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:44.213252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-91af6715-f409-4b70-802f-38243f326d57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41849d511bc14f65b8339629667a9480'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:44.244928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-91af6715-f409-4b70-802f-38243f326d57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41849d511bc14f65b8339629667a9480'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:44.245865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd57780-f0fb-436c-8040-65e73c8846ec None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:44.246861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-91af6715-f409-4b70-802f-38243f326d57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41849d511bc14f65b8339629667a9480'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:44.256239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ff15b-571f-483b-aac8-8c2bbc2c32da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:44.258835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ff15b-571f-483b-aac8-8c2bbc2c32da service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:44.259402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ff15b-571f-483b-aac8-8c2bbc2c32da service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:44.260898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ff15b-571f-483b-aac8-8c2bbc2c32da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:44.262376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-91af6715-f409-4b70-802f-38243f326d57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41849d511bc14f65b8339629667a9480'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:44.264817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-91af6715-f409-4b70-802f-38243f326d57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41849d511bc14f65b8339629667a9480'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:44.282897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ff15b-571f-483b-aac8-8c2bbc2c32da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:44.294698 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ff15b-571f-483b-aac8-8c2bbc2c32da service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:44.299454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 959/1930] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:43 2022] DELETE /identity/v3/users/41849d511bc14f65b8339629667a9480 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:44.301119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fb313-2e79-4bc4-aaf9-8507945c0879 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:44.301410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fb313-2e79-4bc4-aaf9-8507945c0879 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:44.301786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790fb313-2e79-4bc4-aaf9-8507945c0879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:44.324439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 972/1931] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:27:44.341965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-790fb313-2e79-4bc4-aaf9-8507945c0879 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:44.343920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-790fb313-2e79-4bc4-aaf9-8507945c0879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:44.372628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:44.445348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 960/1932] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:44.495426 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ff15b-571f-483b-aac8-8c2bbc2c32da service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:44.496864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-162ff15b-571f-483b-aac8-8c2bbc2c32da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:44.525746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ff15b-571f-483b-aac8-8c2bbc2c32da service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:44.542360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:44.604186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7d2fe-73c4-44cc-8d0b-59a74c380020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:44.605140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7d2fe-73c4-44cc-8d0b-59a74c380020 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:44.605524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7d2fe-73c4-44cc-8d0b-59a74c380020 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:44.605942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7d2fe-73c4-44cc-8d0b-59a74c380020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:44.619705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7d2fe-73c4-44cc-8d0b-59a74c380020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:44.621758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7d2fe-73c4-44cc-8d0b-59a74c380020 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:44.634719 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 973/1933] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:45.781251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fb313-2e79-4bc4-aaf9-8507945c0879 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:45.783478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-790fb313-2e79-4bc4-aaf9-8507945c0879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:45.811335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790fb313-2e79-4bc4-aaf9-8507945c0879 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:45.828729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:45.831158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:45.836473 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:45.836965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] PATH_INFO: `/v3/projects/5e4965cab58d4123b2a1e4fbc7f2c85d` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:45.840243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e4965cab58d4123b2a1e4fbc7f2c85d)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:45.846949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:45.931114 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4965cab58d4123b2a1e4fbc7f2c85d', 'request_id': 'req-ed438661-5700-414e-ae25-1d8f2533e868'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:45.938847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4965cab58d4123b2a1e4fbc7f2c85d', 'request_id': 'req-ed438661-5700-414e-ae25-1d8f2533e868'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:45.950022 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4965cab58d4123b2a1e4fbc7f2c85d', 'request_id': 'req-ed438661-5700-414e-ae25-1d8f2533e868'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:45.968832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] The token cache is being invalidate because project 5e4965cab58d4123b2a1e4fbc7f2c85d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:45.969510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:45.970704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ed438661-5700-414e-ae25-1d8f2533e868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:45.974577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 961/1934] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:45 2022] DELETE /identity/v3/projects/5e4965cab58d4123b2a1e4fbc7f2c85d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:45.977822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf1471-e3c9-447e-a0a8-29ad28153031 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:45.978323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf1471-e3c9-447e-a0a8-29ad28153031 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:45.978499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf1471-e3c9-447e-a0a8-29ad28153031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:45.999102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0aaf1471-e3c9-447e-a0a8-29ad28153031 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:46.001203 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0aaf1471-e3c9-447e-a0a8-29ad28153031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:46.130567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 974/1935] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:46.133942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1b61f-de8d-44d0-aaeb-ce9311891d9c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:46.134313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1b61f-de8d-44d0-aaeb-ce9311891d9c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:46.135310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b1b61f-de8d-44d0-aaeb-ce9311891d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:46.173880 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-13b1b61f-de8d-44d0-aaeb-ce9311891d9c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:46.175533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-13b1b61f-de8d-44d0-aaeb-ce9311891d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:46.194502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 962/1936] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:46.197464 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf1471-e3c9-447e-a0a8-29ad28153031 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:46.210157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf1471-e3c9-447e-a0a8-29ad28153031 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:46.216425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640d53c-182c-4cf6-a9d9-19d18d2ba628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:46.217865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d53c-182c-4cf6-a9d9-19d18d2ba628 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:46.218153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d53c-182c-4cf6-a9d9-19d18d2ba628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:46.218476 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640d53c-182c-4cf6-a9d9-19d18d2ba628 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:46.219744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640d53c-182c-4cf6-a9d9-19d18d2ba628 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:46.220111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640d53c-182c-4cf6-a9d9-19d18d2ba628 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:46.222643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640d53c-182c-4cf6-a9d9-19d18d2ba628 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:46.241150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 975/1937] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:27:46.242883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1b61f-de8d-44d0-aaeb-ce9311891d9c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:46.252420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b1b61f-de8d-44d0-aaeb-ce9311891d9c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:46.257779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14037aad-54cf-472e-aa6a-5baa8584e6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:46.258443 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14037aad-54cf-472e-aa6a-5baa8584e6bb admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:46.258765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14037aad-54cf-472e-aa6a-5baa8584e6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:46.259038 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14037aad-54cf-472e-aa6a-5baa8584e6bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:46.260057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14037aad-54cf-472e-aa6a-5baa8584e6bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:46.261945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14037aad-54cf-472e-aa6a-5baa8584e6bb admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:46.300543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640d53c-182c-4cf6-a9d9-19d18d2ba628 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:46.301462 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 963/1938] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:46 2022] POST /identity/v3/projects => generated 399 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:46.306357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14037aad-54cf-472e-aa6a-5baa8584e6bb admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:46.307017 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5640d53c-182c-4cf6-a9d9-19d18d2ba628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:46.333452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640d53c-182c-4cf6-a9d9-19d18d2ba628 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:46.423280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:46.424800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:46.425008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:46.425289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] PATH_INFO: `/v3/projects/f466b053940e457b96a8e4a40f9a3879` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:46.426825 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f466b053940e457b96a8e4a40f9a3879)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:46.429507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:46.443018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14037aad-54cf-472e-aa6a-5baa8584e6bb admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:46.463689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b106e5-45c2-4e8e-953a-653bafa667a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:46.464798 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b106e5-45c2-4e8e-953a-653bafa667a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:46.465280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b106e5-45c2-4e8e-953a-653bafa667a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:46.467187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b106e5-45c2-4e8e-953a-653bafa667a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:46.468213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b106e5-45c2-4e8e-953a-653bafa667a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:46.472011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b106e5-45c2-4e8e-953a-653bafa667a0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:46.491138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:46.501665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f466b053940e457b96a8e4a40f9a3879', 'request_id': 'req-22825417-0cc8-4a08-98b3-4422e0245c1e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:46.512783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f466b053940e457b96a8e4a40f9a3879', 'request_id': 'req-22825417-0cc8-4a08-98b3-4422e0245c1e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:46.528731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 964/1939] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:46 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:46.532965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b106e5-45c2-4e8e-953a-653bafa667a0 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:46.539740 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f466b053940e457b96a8e4a40f9a3879', 'request_id': 'req-22825417-0cc8-4a08-98b3-4422e0245c1e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:46.551537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:46.559372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b106e5-45c2-4e8e-953a-653bafa667a0 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:46.564996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] The token cache is being invalidate because project f466b053940e457b96a8e4a40f9a3879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:46.566774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:46.569608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:46.571998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 976/1940] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:46 2022] DELETE /identity/v3/projects/f466b053940e457b96a8e4a40f9a3879 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:46.576243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:46.581308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:46.749407 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee815ed-0cba-49bf-8e40-667862eee262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:46.750125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee815ed-0cba-49bf-8e40-667862eee262 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:46.750412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee815ed-0cba-49bf-8e40-667862eee262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:46.752785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee815ed-0cba-49bf-8e40-667862eee262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:46.754981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee815ed-0cba-49bf-8e40-667862eee262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:46.757896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee815ed-0cba-49bf-8e40-667862eee262 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:46.762509 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22825417-0cc8-4a08-98b3-4422e0245c1e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:46.775205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eadd29-41d2-4788-a2c4-d99fa79f0015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:46.783153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eadd29-41d2-4788-a2c4-d99fa79f0015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:46.783482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eadd29-41d2-4788-a2c4-d99fa79f0015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:46.784847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eadd29-41d2-4788-a2c4-d99fa79f0015 admin admin] PATH_INFO: `/v3/users/4594ada515ef4fad9d28e4f3ee220843` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:46.790653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eadd29-41d2-4788-a2c4-d99fa79f0015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4594ada515ef4fad9d28e4f3ee220843)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:46.798436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eadd29-41d2-4788-a2c4-d99fa79f0015 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:46.805375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 965/1941] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:27:46.823545 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee815ed-0cba-49bf-8e40-667862eee262 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:46.830118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-eee815ed-0cba-49bf-8e40-667862eee262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:46.897679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee815ed-0cba-49bf-8e40-667862eee262 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:46.910488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebcc8c-be43-4f11-8937-a0ada8241198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:46.911182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebcc8c-be43-4f11-8937-a0ada8241198 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:46.912622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebcc8c-be43-4f11-8937-a0ada8241198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:46.912902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebcc8c-be43-4f11-8937-a0ada8241198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:46.913962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ebcc8c-be43-4f11-8937-a0ada8241198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:46.917129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ebcc8c-be43-4f11-8937-a0ada8241198 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:46.953469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 966/1942] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:46.955545 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-77eadd29-41d2-4788-a2c4-d99fa79f0015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4594ada515ef4fad9d28e4f3ee220843'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:46.956421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebcc8c-be43-4f11-8937-a0ada8241198 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:46.972138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-77eadd29-41d2-4788-a2c4-d99fa79f0015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4594ada515ef4fad9d28e4f3ee220843'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:46.972931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-77eadd29-41d2-4788-a2c4-d99fa79f0015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4594ada515ef4fad9d28e4f3ee220843'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:46.978906 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebcc8c-be43-4f11-8937-a0ada8241198 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:46.981601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-77eadd29-41d2-4788-a2c4-d99fa79f0015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4594ada515ef4fad9d28e4f3ee220843'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:46.982905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-77eadd29-41d2-4788-a2c4-d99fa79f0015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4594ada515ef4fad9d28e4f3ee220843'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:46.991007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf03b6b-b949-4e68-b4c2-17e24b0f15a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:46.991916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf03b6b-b949-4e68-b4c2-17e24b0f15a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:46.992198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf03b6b-b949-4e68-b4c2-17e24b0f15a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:46.992469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf03b6b-b949-4e68-b4c2-17e24b0f15a2 admin admin] PATH_INFO: `/v3/projects/e39ddd6683c046dcb9f863b2a45ffba7/users/a73b234e7f214a02bd3addc6725b734a/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:47.002326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 977/1943] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:46 2022] DELETE /identity/v3/users/4594ada515ef4fad9d28e4f3ee220843 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:47.029893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf03b6b-b949-4e68-b4c2-17e24b0f15a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39ddd6683c046dcb9f863b2a45ffba7, user_id=a73b234e7f214a02bd3addc6725b734a, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:47.035737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf03b6b-b949-4e68-b4c2-17e24b0f15a2 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:47.091412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 967/1944] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:46 2022] PUT /identity/v3/projects/e39ddd6683c046dcb9f863b2a45ffba7/users/a73b234e7f214a02bd3addc6725b734a/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:47.104770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ed974-bd2f-423c-8b3e-ee18abf411c3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:47.105340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ed974-bd2f-423c-8b3e-ee18abf411c3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:47.105753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ed974-bd2f-423c-8b3e-ee18abf411c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:47.165028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-302ed974-bd2f-423c-8b3e-ee18abf411c3 None None] MFA Rules not processed for user `a73b234e7f214a02bd3addc6725b734a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:47.166822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-302ed974-bd2f-423c-8b3e-ee18abf411c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:47.243209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:47.278405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 978/1945] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:47.290867 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf03b6b-b949-4e68-b4c2-17e24b0f15a2 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:47.293158 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-bbf03b6b-b949-4e68-b4c2-17e24b0f15a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:47.402214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf03b6b-b949-4e68-b4c2-17e24b0f15a2 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:47.404580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e787f-2ee0-4a4b-b323-beede8503f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:47.405386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e787f-2ee0-4a4b-b323-beede8503f1e service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:47.405768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e787f-2ee0-4a4b-b323-beede8503f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:47.406150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e787f-2ee0-4a4b-b323-beede8503f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:47.420526 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e787f-2ee0-4a4b-b323-beede8503f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:47.422413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e787f-2ee0-4a4b-b323-beede8503f1e service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:47.470497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 968/1946] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:47.691739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ed974-bd2f-423c-8b3e-ee18abf411c3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:47.693253 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-302ed974-bd2f-423c-8b3e-ee18abf411c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:47.779428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ed974-bd2f-423c-8b3e-ee18abf411c3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:47.784551 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:47.785420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:47.785703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:47.786000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] PATH_INFO: `/v3/projects/f2521b0a2aa040389b39f5a7e88734fb` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:47.787982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2521b0a2aa040389b39f5a7e88734fb)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:47.790554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:47.869543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2521b0a2aa040389b39f5a7e88734fb', 'request_id': 'req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:47.877596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2521b0a2aa040389b39f5a7e88734fb', 'request_id': 'req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:47.892555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2521b0a2aa040389b39f5a7e88734fb', 'request_id': 'req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:47.920629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] The token cache is being invalidate because project f2521b0a2aa040389b39f5a7e88734fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:47.921716 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:47.922783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-3b882592-d749-4ad6-9c2d-dd2cfee4b7b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:47.924531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 979/1947] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:47 2022] DELETE /identity/v3/projects/f2521b0a2aa040389b39f5a7e88734fb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:47.930286 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be88c5d-b3cb-47dc-90d2-d5ba61400c3e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:47.931187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be88c5d-b3cb-47dc-90d2-d5ba61400c3e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:47.932213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be88c5d-b3cb-47dc-90d2-d5ba61400c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:47.954561 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-6be88c5d-b3cb-47dc-90d2-d5ba61400c3e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:47.955980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6be88c5d-b3cb-47dc-90d2-d5ba61400c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:48.074336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 969/1948] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:48.083144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dde4e-4fea-47bb-9bab-846ac321ccfe None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:48.083644 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dde4e-4fea-47bb-9bab-846ac321ccfe None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:48.083737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626dde4e-4fea-47bb-9bab-846ac321ccfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:48.139456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-626dde4e-4fea-47bb-9bab-846ac321ccfe None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:48.140714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-626dde4e-4fea-47bb-9bab-846ac321ccfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:48.164221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 980/1949] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:48.169980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be88c5d-b3cb-47dc-90d2-d5ba61400c3e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:48.187961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be88c5d-b3cb-47dc-90d2-d5ba61400c3e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:48.202931 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d9cca-659e-4757-ab49-1a2c64b34691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:48.204285 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d9cca-659e-4757-ab49-1a2c64b34691 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:48.204602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d9cca-659e-4757-ab49-1a2c64b34691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:48.204902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d9cca-659e-4757-ab49-1a2c64b34691 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:48.215735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d9cca-659e-4757-ab49-1a2c64b34691 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:48.216195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d9cca-659e-4757-ab49-1a2c64b34691 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:48.225953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d9cca-659e-4757-ab49-1a2c64b34691 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:48.234160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:48.269391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 970/1950] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:27:48.270813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626dde4e-4fea-47bb-9bab-846ac321ccfe None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:48.285693 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626dde4e-4fea-47bb-9bab-846ac321ccfe None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:48.309668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97fc2e-ae75-4bf7-8890-3c3f63e63121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:48.310902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97fc2e-ae75-4bf7-8890-3c3f63e63121 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:48.312809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97fc2e-ae75-4bf7-8890-3c3f63e63121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:48.313199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97fc2e-ae75-4bf7-8890-3c3f63e63121 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:48.315234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97fc2e-ae75-4bf7-8890-3c3f63e63121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:48.323077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97fc2e-ae75-4bf7-8890-3c3f63e63121 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:48.433502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d9cca-659e-4757-ab49-1a2c64b34691 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:48.438406 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 981/1951] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:48 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:48.446764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d9cca-659e-4757-ab49-1a2c64b34691 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:48.486014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d667878-d3b9-4160-931e-9d1c03218a4f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:48.486310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d667878-d3b9-4160-931e-9d1c03218a4f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:48.486629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d667878-d3b9-4160-931e-9d1c03218a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:48.521804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0d667878-d3b9-4160-931e-9d1c03218a4f None None] MFA Rules not processed for user `b876d4f582d0478093e0bb22fbf07ecf`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:48.523670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0d667878-d3b9-4160-931e-9d1c03218a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:48.540779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1206e22-6b87-491a-8293-131c28d18890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:48.542054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1206e22-6b87-491a-8293-131c28d18890 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:48.542418 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1206e22-6b87-491a-8293-131c28d18890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:48.542927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1206e22-6b87-491a-8293-131c28d18890 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:48.543876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1206e22-6b87-491a-8293-131c28d18890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:48.547593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1206e22-6b87-491a-8293-131c28d18890 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:48.574874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:48.582737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 971/1952] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:48 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:27:48.587396 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1206e22-6b87-491a-8293-131c28d18890 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:48.602416 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1206e22-6b87-491a-8293-131c28d18890 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:48.609729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92408d3-3877-4fa0-a4f7-ca923a93f408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:48.610806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92408d3-3877-4fa0-a4f7-ca923a93f408 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:48.610806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92408d3-3877-4fa0-a4f7-ca923a93f408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:48.610998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92408d3-3877-4fa0-a4f7-ca923a93f408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:48.611652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92408d3-3877-4fa0-a4f7-ca923a93f408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:48.617649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92408d3-3877-4fa0-a4f7-ca923a93f408 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:48.631486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 972/1953] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:27:48.636418 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92408d3-3877-4fa0-a4f7-ca923a93f408 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:48.648466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92408d3-3877-4fa0-a4f7-ca923a93f408 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:48.649303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 982/1954] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:27:48.655115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c624a2-4936-49b0-9652-62806409fe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:48.655787 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c624a2-4936-49b0-9652-62806409fe7b admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:48.656072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c624a2-4936-49b0-9652-62806409fe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:48.656345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c624a2-4936-49b0-9652-62806409fe7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:48.657079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c624a2-4936-49b0-9652-62806409fe7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:48.658812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c624a2-4936-49b0-9652-62806409fe7b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:48.665150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d667878-d3b9-4160-931e-9d1c03218a4f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:48.666815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0d667878-d3b9-4160-931e-9d1c03218a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:48.672022 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 973/1955] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:48.676382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c624a2-4936-49b0-9652-62806409fe7b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:48.687330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c624a2-4936-49b0-9652-62806409fe7b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:48.692231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237018d-5f00-4065-940d-cc1f8e3abb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:48.693258 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237018d-5f00-4065-940d-cc1f8e3abb51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:48.693572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237018d-5f00-4065-940d-cc1f8e3abb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:48.693847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237018d-5f00-4065-940d-cc1f8e3abb51 admin admin] PATH_INFO: `/v3/projects/516fe8f22188479db1b940bc682bf5a0/users/188d26a3df14484687ff57da147171aa/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:48.709983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237018d-5f00-4065-940d-cc1f8e3abb51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=516fe8f22188479db1b940bc682bf5a0, user_id=188d26a3df14484687ff57da147171aa, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:48.713047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237018d-5f00-4065-940d-cc1f8e3abb51 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:48.731545 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 974/1956] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:48 2022] PUT /identity/v3/projects/516fe8f22188479db1b940bc682bf5a0/users/188d26a3df14484687ff57da147171aa/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:48.736708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205b1f5-da12-406f-83ca-458f7353182a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:48.737715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205b1f5-da12-406f-83ca-458f7353182a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:48.738075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205b1f5-da12-406f-83ca-458f7353182a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:48.740612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d667878-d3b9-4160-931e-9d1c03218a4f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:48.759701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-8205b1f5-da12-406f-83ca-458f7353182a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:48.761128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8205b1f5-da12-406f-83ca-458f7353182a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:48.796789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec865683-3c4d-4bf4-b200-f3d542104060 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13121a263c31433dbd4a5082b4bf3bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:48.797622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec865683-3c4d-4bf4-b200-f3d542104060 service glance] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:48.797915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec865683-3c4d-4bf4-b200-f3d542104060 service glance] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:48.798199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec865683-3c4d-4bf4-b200-f3d542104060 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:48.800559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ec865683-3c4d-4bf4-b200-f3d542104060 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:48.835325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 975/1957] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:48.846008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205b1f5-da12-406f-83ca-458f7353182a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:48.847554 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8205b1f5-da12-406f-83ca-458f7353182a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:48.873235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec865683-3c4d-4bf4-b200-f3d542104060 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:48.874843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec865683-3c4d-4bf4-b200-f3d542104060 service glance] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:48.888848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 983/1958] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:27:48.895943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:48.915907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205b1f5-da12-406f-83ca-458f7353182a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:48.920106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fb0de-7004-4ced-ac46-8e53ac9394a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:48.920764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fb0de-7004-4ced-ac46-8e53ac9394a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:48.921047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fb0de-7004-4ced-ac46-8e53ac9394a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:48.921321 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fb0de-7004-4ced-ac46-8e53ac9394a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:48.933186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fb0de-7004-4ced-ac46-8e53ac9394a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:48.935019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fb0de-7004-4ced-ac46-8e53ac9394a7 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:48.951915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 976/1959] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:49.016168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b0ff6-c24f-4cef-be26-88337ff11b3f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:49.016468 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b0ff6-c24f-4cef-be26-88337ff11b3f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:49.016943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b0ff6-c24f-4cef-be26-88337ff11b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:49.036372 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f36b0ff6-c24f-4cef-be26-88337ff11b3f None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:49.037537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f36b0ff6-c24f-4cef-be26-88337ff11b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:49.080057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:49.112408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 984/1960] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:49.169836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fb0de-7004-4ced-ac46-8e53ac9394a7 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:49.171781 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f36fb0de-7004-4ced-ac46-8e53ac9394a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:49.277100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fb0de-7004-4ced-ac46-8e53ac9394a7 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:49.282283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603f0e3-c962-4d56-b8a3-08e201006cf9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:49.282959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603f0e3-c962-4d56-b8a3-08e201006cf9 service swift] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:49.283332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603f0e3-c962-4d56-b8a3-08e201006cf9 service swift] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:49.283508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603f0e3-c962-4d56-b8a3-08e201006cf9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:49.296253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603f0e3-c962-4d56-b8a3-08e201006cf9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:49.298504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603f0e3-c962-4d56-b8a3-08e201006cf9 service swift] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:49.314319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 977/1961] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:27:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:27:50.099729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8706101-98a7-43a5-b267-052fada246f3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:50.099729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8706101-98a7-43a5-b267-052fada246f3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:50.100146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8706101-98a7-43a5-b267-052fada246f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:50.143441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a8706101-98a7-43a5-b267-052fada246f3 None None] MFA Rules not processed for user `188d26a3df14484687ff57da147171aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:50.144656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a8706101-98a7-43a5-b267-052fada246f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:50.240072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 985/1962] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:50.244841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603f0e3-c962-4d56-b8a3-08e201006cf9 service swift] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:50.273948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603f0e3-c962-4d56-b8a3-08e201006cf9 service swift] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:50.281291 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8954d9-4f9d-4346-8c6e-d8cdbca12c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:50.281979 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8954d9-4f9d-4346-8c6e-d8cdbca12c0c admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:50.282261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8954d9-4f9d-4346-8c6e-d8cdbca12c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:50.282589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8954d9-4f9d-4346-8c6e-d8cdbca12c0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:50.283290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8954d9-4f9d-4346-8c6e-d8cdbca12c0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:50.285289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8954d9-4f9d-4346-8c6e-d8cdbca12c0c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:50.324310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 978/1963] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:50 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:50.328473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8706101-98a7-43a5-b267-052fada246f3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:50.341860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8706101-98a7-43a5-b267-052fada246f3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:50.419844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277987d-d337-4314-a443-334d0d3c96f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:50.420506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277987d-d337-4314-a443-334d0d3c96f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:50.420790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277987d-d337-4314-a443-334d0d3c96f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:50.421111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277987d-d337-4314-a443-334d0d3c96f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:50.422062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277987d-d337-4314-a443-334d0d3c96f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:50.424412 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277987d-d337-4314-a443-334d0d3c96f3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:50.449187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 986/1964] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:50 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:27:50.453066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8954d9-4f9d-4346-8c6e-d8cdbca12c0c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:50.465912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8954d9-4f9d-4346-8c6e-d8cdbca12c0c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:50.471457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11137f9-dfb4-4963-ab02-8b9d64079b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:50.472252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11137f9-dfb4-4963-ab02-8b9d64079b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:50.472543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11137f9-dfb4-4963-ab02-8b9d64079b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:50.472997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11137f9-dfb4-4963-ab02-8b9d64079b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:50.474109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11137f9-dfb4-4963-ab02-8b9d64079b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:50.476307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11137f9-dfb4-4963-ab02-8b9d64079b7a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:50.494990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 979/1965] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:50.495756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277987d-d337-4314-a443-334d0d3c96f3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:50.516307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277987d-d337-4314-a443-334d0d3c96f3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:50.528000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999bd7d-fce9-44a6-b16d-19fe6dcfe91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:50.531536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999bd7d-fce9-44a6-b16d-19fe6dcfe91f admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:50.531822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999bd7d-fce9-44a6-b16d-19fe6dcfe91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:50.532115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999bd7d-fce9-44a6-b16d-19fe6dcfe91f admin admin] PATH_INFO: `/v3/projects/20293df30c384ad9a8b421f89d79c920/users/3c8066e46e624db1bed5bade77fed011/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:50.574475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999bd7d-fce9-44a6-b16d-19fe6dcfe91f admin admin] RBAC: Authorizing `identity:create_grant(project_id=20293df30c384ad9a8b421f89d79c920, user_id=3c8066e46e624db1bed5bade77fed011, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:50.577210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999bd7d-fce9-44a6-b16d-19fe6dcfe91f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:50.595303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 987/1966] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:50 2022] PUT /identity/v3/projects/20293df30c384ad9a8b421f89d79c920/users/3c8066e46e624db1bed5bade77fed011/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:50.599333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11137f9-dfb4-4963-ab02-8b9d64079b7a admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:50.610336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11137f9-dfb4-4963-ab02-8b9d64079b7a admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:50.715359 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33807a63-f889-44e7-91e5-9ef1cd40c3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:50.716049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33807a63-f889-44e7-91e5-9ef1cd40c3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:50.716334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33807a63-f889-44e7-91e5-9ef1cd40c3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:50.716609 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33807a63-f889-44e7-91e5-9ef1cd40c3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:50.717352 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33807a63-f889-44e7-91e5-9ef1cd40c3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:50.719633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33807a63-f889-44e7-91e5-9ef1cd40c3f2 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:50.738677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 980/1967] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:50.740472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999bd7d-fce9-44a6-b16d-19fe6dcfe91f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:50.751400 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999bd7d-fce9-44a6-b16d-19fe6dcfe91f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:50.757720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf8457-d57c-4749-8ca1-c7fe97e8674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:50.758907 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf8457-d57c-4749-8ca1-c7fe97e8674c admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:50.764269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf8457-d57c-4749-8ca1-c7fe97e8674c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:50.764797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf8457-d57c-4749-8ca1-c7fe97e8674c admin admin] PATH_INFO: `/v3/projects/20293df30c384ad9a8b421f89d79c920/users/3c8066e46e624db1bed5bade77fed011/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:50.767053 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf8457-d57c-4749-8ca1-c7fe97e8674c admin admin] RBAC: Authorizing `identity:create_grant(project_id=20293df30c384ad9a8b421f89d79c920, user_id=3c8066e46e624db1bed5bade77fed011, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:50.769930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf8457-d57c-4749-8ca1-c7fe97e8674c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:50.789905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 988/1968] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:50 2022] PUT /identity/v3/projects/20293df30c384ad9a8b421f89d79c920/users/3c8066e46e624db1bed5bade77fed011/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:51.824317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d1285-f8d4-44d5-b26a-cd0dd10ca52a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:51.825320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d1285-f8d4-44d5-b26a-cd0dd10ca52a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:51.825759 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d1285-f8d4-44d5-b26a-cd0dd10ca52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:51.882010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2e3d1285-f8d4-44d5-b26a-cd0dd10ca52a None None] MFA Rules not processed for user `3c8066e46e624db1bed5bade77fed011`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:51.883757 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2e3d1285-f8d4-44d5-b26a-cd0dd10ca52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:51.923318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:52.016511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf8457-d57c-4749-8ca1-c7fe97e8674c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:52.017816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 981/1969] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:52.018635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-addf8457-d57c-4749-8ca1-c7fe97e8674c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:52.049708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:52.087993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf8457-d57c-4749-8ca1-c7fe97e8674c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:52.094036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271be6d4-c7d3-42bb-ba1d-98a74b9ead53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:52.095338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271be6d4-c7d3-42bb-ba1d-98a74b9ead53 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:52.095621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271be6d4-c7d3-42bb-ba1d-98a74b9ead53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:52.095934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271be6d4-c7d3-42bb-ba1d-98a74b9ead53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:52.201786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271be6d4-c7d3-42bb-ba1d-98a74b9ead53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:52.205949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271be6d4-c7d3-42bb-ba1d-98a74b9ead53 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:52.217746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 989/1970] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:27:53.812116 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d1285-f8d4-44d5-b26a-cd0dd10ca52a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:53.813740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2e3d1285-f8d4-44d5-b26a-cd0dd10ca52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:53.844652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d1285-f8d4-44d5-b26a-cd0dd10ca52a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:53.863907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c5e71-99dc-412b-b555-bb31bc6ffc34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:53.870750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c5e71-99dc-412b-b555-bb31bc6ffc34 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:53.871049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c5e71-99dc-412b-b555-bb31bc6ffc34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:53.871350 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c5e71-99dc-412b-b555-bb31bc6ffc34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:53.904257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c5e71-99dc-412b-b555-bb31bc6ffc34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:53.905925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c5e71-99dc-412b-b555-bb31bc6ffc34 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:53.923846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 982/1971] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:55.052205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6eb8b3-4aaf-4d05-b8bc-adcb54c1bbde None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:55.053408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6eb8b3-4aaf-4d05-b8bc-adcb54c1bbde None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:55.053408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6eb8b3-4aaf-4d05-b8bc-adcb54c1bbde None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:55.055074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 990/1972] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:27:55.070345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0930a-632e-4c74-8e6f-c23f8ee93317 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:55.070950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0930a-632e-4c74-8e6f-c23f8ee93317 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:55.071164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0930a-632e-4c74-8e6f-c23f8ee93317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:55.120219 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c1e0930a-632e-4c74-8e6f-c23f8ee93317 None None] MFA Rules not processed for user `50625c2b38ed41698ba6cccdb12730ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:55.122615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c1e0930a-632e-4c74-8e6f-c23f8ee93317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:55.211385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6eb8b3-4aaf-4d05-b8bc-adcb54c1bbde None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:55.216994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6f6eb8b3-4aaf-4d05-b8bc-adcb54c1bbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:55.264126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 983/1973] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:27:55.281208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0930a-632e-4c74-8e6f-c23f8ee93317 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:55.284323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c1e0930a-632e-4c74-8e6f-c23f8ee93317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:55.312454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0930a-632e-4c74-8e6f-c23f8ee93317 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:55.320992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e83e76-7584-4c44-924c-e2e8199256dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:55.321869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e83e76-7584-4c44-924c-e2e8199256dd service placement] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:55.322163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e83e76-7584-4c44-924c-e2e8199256dd service placement] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:55.322447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e83e76-7584-4c44-924c-e2e8199256dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:55.339624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e83e76-7584-4c44-924c-e2e8199256dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:55.341381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e83e76-7584-4c44-924c-e2e8199256dd service placement] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:55.370998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 984/1974] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:55.396968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6eb8b3-4aaf-4d05-b8bc-adcb54c1bbde None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:55.413996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062bf29-6961-4c4c-b88f-7ec080dec3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:55.414599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062bf29-6961-4c4c-b88f-7ec080dec3e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:55.415033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062bf29-6961-4c4c-b88f-7ec080dec3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:55.415388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062bf29-6961-4c4c-b88f-7ec080dec3e4 admin admin] PATH_INFO: `/v3/users/bdd62219456547db82c2f5416d4f44f6` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:55.417310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062bf29-6961-4c4c-b88f-7ec080dec3e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdd62219456547db82c2f5416d4f44f6)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:55.423618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062bf29-6961-4c4c-b88f-7ec080dec3e4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:55.529598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a062bf29-6961-4c4c-b88f-7ec080dec3e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdd62219456547db82c2f5416d4f44f6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:55.561630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a062bf29-6961-4c4c-b88f-7ec080dec3e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdd62219456547db82c2f5416d4f44f6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:55.562458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a062bf29-6961-4c4c-b88f-7ec080dec3e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdd62219456547db82c2f5416d4f44f6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:55.573446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a062bf29-6961-4c4c-b88f-7ec080dec3e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdd62219456547db82c2f5416d4f44f6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:55.574208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a062bf29-6961-4c4c-b88f-7ec080dec3e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdd62219456547db82c2f5416d4f44f6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:55.581621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:55.600986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 991/1975] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:55 2022] DELETE /identity/v3/users/bdd62219456547db82c2f5416d4f44f6 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:55.934256 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e83e76-7584-4c44-924c-e2e8199256dd service placement] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:55.960530 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e83e76-7584-4c44-924c-e2e8199256dd service placement] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:56.046465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:56.047467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.047826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.048177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] PATH_INFO: `/v3/projects/683acaeef18d43b4ac3daeb60e3a8aea` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.049409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] RBAC: Authorizing `identity:delete_project(project_id=683acaeef18d43b4ac3daeb60e3a8aea)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:56.051397 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:56.099753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:56.120318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683acaeef18d43b4ac3daeb60e3a8aea', 'request_id': 'req-1eb0ae09-1a03-4828-ba62-0bf58300ebac'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:56.133700 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683acaeef18d43b4ac3daeb60e3a8aea', 'request_id': 'req-1eb0ae09-1a03-4828-ba62-0bf58300ebac'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:56.146314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683acaeef18d43b4ac3daeb60e3a8aea', 'request_id': 'req-1eb0ae09-1a03-4828-ba62-0bf58300ebac'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:56.163434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] The token cache is being invalidate because project 683acaeef18d43b4ac3daeb60e3a8aea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:56.163829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:56.164541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1eb0ae09-1a03-4828-ba62-0bf58300ebac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:56.167728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 985/1976] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:55 2022] DELETE /identity/v3/projects/683acaeef18d43b4ac3daeb60e3a8aea => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:56.172279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1d9eb-9da2-41e0-aeed-ec856ce1e014 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.172824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1d9eb-9da2-41e0-aeed-ec856ce1e014 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.172824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1d9eb-9da2-41e0-aeed-ec856ce1e014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.204251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-33e1d9eb-9da2-41e0-aeed-ec856ce1e014 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:56.205664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-33e1d9eb-9da2-41e0-aeed-ec856ce1e014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:56.271570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 992/1977] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:56.282942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3feb8e-cf15-4413-b1a7-cc6a9cd0d85b None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.283229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3feb8e-cf15-4413-b1a7-cc6a9cd0d85b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.283527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3feb8e-cf15-4413-b1a7-cc6a9cd0d85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.309751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2f3feb8e-cf15-4413-b1a7-cc6a9cd0d85b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:56.310997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2f3feb8e-cf15-4413-b1a7-cc6a9cd0d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:56.324411 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 986/1978] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:27:56.328481 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1d9eb-9da2-41e0-aeed-ec856ce1e014 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:56.339366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e1d9eb-9da2-41e0-aeed-ec856ce1e014 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:56.345404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180ba99-d96b-4850-b976-ec916e8abf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:56.346403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180ba99-d96b-4850-b976-ec916e8abf30 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.346834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180ba99-d96b-4850-b976-ec916e8abf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.347507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180ba99-d96b-4850-b976-ec916e8abf30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.348109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180ba99-d96b-4850-b976-ec916e8abf30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:56.350304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180ba99-d96b-4850-b976-ec916e8abf30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:56.352056 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180ba99-d96b-4850-b976-ec916e8abf30 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:56.366573 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 993/1979] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:27:56.370914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3feb8e-cf15-4413-b1a7-cc6a9cd0d85b None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:56.382427 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3feb8e-cf15-4413-b1a7-cc6a9cd0d85b None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:56.392199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180ba99-d96b-4850-b976-ec916e8abf30 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:56.394028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2da12b-9da7-46fb-9d28-6c4d383db672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:56.394760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2da12b-9da7-46fb-9d28-6c4d383db672 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.394892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e180ba99-d96b-4850-b976-ec916e8abf30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:56.395052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2da12b-9da7-46fb-9d28-6c4d383db672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.398055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2da12b-9da7-46fb-9d28-6c4d383db672 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.398559 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2da12b-9da7-46fb-9d28-6c4d383db672 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:56.402325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2da12b-9da7-46fb-9d28-6c4d383db672 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:56.412332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180ba99-d96b-4850-b976-ec916e8abf30 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:56.417749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8749102-6f87-428b-aab9-2dc8010cef7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:56.421052 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8749102-6f87-428b-aab9-2dc8010cef7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.422304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8749102-6f87-428b-aab9-2dc8010cef7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.422304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8749102-6f87-428b-aab9-2dc8010cef7f admin admin] PATH_INFO: `/v3/users/a73b234e7f214a02bd3addc6725b734a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.423403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8749102-6f87-428b-aab9-2dc8010cef7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a73b234e7f214a02bd3addc6725b734a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:56.425460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8749102-6f87-428b-aab9-2dc8010cef7f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:56.457967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 987/1980] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:56 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:56.462196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2da12b-9da7-46fb-9d28-6c4d383db672 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:56.524409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8749102-6f87-428b-aab9-2dc8010cef7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a73b234e7f214a02bd3addc6725b734a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:56.542676 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8749102-6f87-428b-aab9-2dc8010cef7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a73b234e7f214a02bd3addc6725b734a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:56.543418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8749102-6f87-428b-aab9-2dc8010cef7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a73b234e7f214a02bd3addc6725b734a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:56.550945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8749102-6f87-428b-aab9-2dc8010cef7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a73b234e7f214a02bd3addc6725b734a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:56.551673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8749102-6f87-428b-aab9-2dc8010cef7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a73b234e7f214a02bd3addc6725b734a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:56.558671 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2da12b-9da7-46fb-9d28-6c4d383db672 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:56.566496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2084eb0-0094-4fb7-8032-1f74c665b645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:56.567347 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2084eb0-0094-4fb7-8032-1f74c665b645 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.567735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2084eb0-0094-4fb7-8032-1f74c665b645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.568641 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2084eb0-0094-4fb7-8032-1f74c665b645 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.570718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2084eb0-0094-4fb7-8032-1f74c665b645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:56.571204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 994/1981] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:27:56 2022] DELETE /identity/v3/users/a73b234e7f214a02bd3addc6725b734a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:27:56.571871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2084eb0-0094-4fb7-8032-1f74c665b645 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:56.574058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db93d1-a8c0-4dcb-a714-ff735801672d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.574354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db93d1-a8c0-4dcb-a714-ff735801672d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.574860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db93d1-a8c0-4dcb-a714-ff735801672d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.596321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-89db93d1-a8c0-4dcb-a714-ff735801672d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:56.597600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-89db93d1-a8c0-4dcb-a714-ff735801672d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:56.599477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 988/1982] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:56 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:27:56.603430 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2084eb0-0094-4fb7-8032-1f74c665b645 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:56.606708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e2084eb0-0094-4fb7-8032-1f74c665b645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:56.622783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2084eb0-0094-4fb7-8032-1f74c665b645 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:56.665882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:56.687287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 995/1983] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:27:56.696490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db93d1-a8c0-4dcb-a714-ff735801672d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:56.699483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-89db93d1-a8c0-4dcb-a714-ff735801672d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:56.716953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22067e-dbc4-4d2d-a47a-445048a6fc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:56.717798 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22067e-dbc4-4d2d-a47a-445048a6fc1f admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.718177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22067e-dbc4-4d2d-a47a-445048a6fc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.718639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22067e-dbc4-4d2d-a47a-445048a6fc1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.719428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22067e-dbc4-4d2d-a47a-445048a6fc1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:56.721278 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22067e-dbc4-4d2d-a47a-445048a6fc1f admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:56.738574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 989/1984] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:27:56.741611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22067e-dbc4-4d2d-a47a-445048a6fc1f admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:56.756809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22067e-dbc4-4d2d-a47a-445048a6fc1f admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:56.764799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a59589-8a84-494a-a173-27d0555af7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:56.765398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59589-8a84-494a-a173-27d0555af7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.765685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59589-8a84-494a-a173-27d0555af7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.765960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59589-8a84-494a-a173-27d0555af7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.766723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a59589-8a84-494a-a173-27d0555af7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:56.768348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a59589-8a84-494a-a173-27d0555af7a2 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:56.779969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db93d1-a8c0-4dcb-a714-ff735801672d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:56.780635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 990/1985] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:27:56.785609 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a59589-8a84-494a-a173-27d0555af7a2 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:56.786202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862ba34-8cba-4af2-b486-d9664c42018a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:56.787432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862ba34-8cba-4af2-b486-d9664c42018a service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.787712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862ba34-8cba-4af2-b486-d9664c42018a service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.787985 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862ba34-8cba-4af2-b486-d9664c42018a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.799143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a59589-8a84-494a-a173-27d0555af7a2 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:56.799345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862ba34-8cba-4af2-b486-d9664c42018a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:56.800648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862ba34-8cba-4af2-b486-d9664c42018a service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:56.806491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26c259-478d-45a9-80ab-1a40f34f6bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:56.807428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c259-478d-45a9-80ab-1a40f34f6bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.807768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c259-478d-45a9-80ab-1a40f34f6bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.807990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26c259-478d-45a9-80ab-1a40f34f6bf4 admin admin] PATH_INFO: `/v3/projects/10c35bb9eda54d509901fab7285c40e0/users/3dc72ca8fa3742e9a7d2e8bae7b992ae/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.811685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:56.814578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 996/1986] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:56.838072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26c259-478d-45a9-80ab-1a40f34f6bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c35bb9eda54d509901fab7285c40e0, user_id=3dc72ca8fa3742e9a7d2e8bae7b992ae, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:56.841875 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26c259-478d-45a9-80ab-1a40f34f6bf4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:56.866486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 991/1987] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:56 2022] PUT /identity/v3/projects/10c35bb9eda54d509901fab7285c40e0/users/3dc72ca8fa3742e9a7d2e8bae7b992ae/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:56.876357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9fa13-58be-4c3d-a6f0-12258aaa244b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:56.876732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9fa13-58be-4c3d-a6f0-12258aaa244b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:56.877115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9fa13-58be-4c3d-a6f0-12258aaa244b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:56.923791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3fe9fa13-58be-4c3d-a6f0-12258aaa244b None None] MFA Rules not processed for user `3dc72ca8fa3742e9a7d2e8bae7b992ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:56.925500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3fe9fa13-58be-4c3d-a6f0-12258aaa244b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:57.030703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 997/1988] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:57.041073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26c259-478d-45a9-80ab-1a40f34f6bf4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:57.079797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5e26c259-478d-45a9-80ab-1a40f34f6bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:57.264246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26c259-478d-45a9-80ab-1a40f34f6bf4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:57.273093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea8dee-0ff6-40f8-ac5f-edd804a22641 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:57.273907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea8dee-0ff6-40f8-ac5f-edd804a22641 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:57.274094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea8dee-0ff6-40f8-ac5f-edd804a22641 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:57.274510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea8dee-0ff6-40f8-ac5f-edd804a22641 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:57.301358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea8dee-0ff6-40f8-ac5f-edd804a22641 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:57.301358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea8dee-0ff6-40f8-ac5f-edd804a22641 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:57.301982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9fa13-58be-4c3d-a6f0-12258aaa244b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:57.302453 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3fe9fa13-58be-4c3d-a6f0-12258aaa244b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:57.306854 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 992/1989] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:27:57.416421 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe9fa13-58be-4c3d-a6f0-12258aaa244b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:57.423594 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:57.424966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:57.425365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:57.425610 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] PATH_INFO: `/v3/projects/e39ddd6683c046dcb9f863b2a45ffba7` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:57.428679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39ddd6683c046dcb9f863b2a45ffba7)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:57.431322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:57.490333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:57.519298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39ddd6683c046dcb9f863b2a45ffba7', 'request_id': 'req-6fb60558-1916-4450-bf0f-6febd1df5f0f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:57.532162 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39ddd6683c046dcb9f863b2a45ffba7', 'request_id': 'req-6fb60558-1916-4450-bf0f-6febd1df5f0f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:57.560073 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39ddd6683c046dcb9f863b2a45ffba7', 'request_id': 'req-6fb60558-1916-4450-bf0f-6febd1df5f0f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:57.594440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] The token cache is being invalidate because project e39ddd6683c046dcb9f863b2a45ffba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:27:57.595001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:57.596177 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6fb60558-1916-4450-bf0f-6febd1df5f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:27:57.600738 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 998/1990] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:27:57 2022] DELETE /identity/v3/projects/e39ddd6683c046dcb9f863b2a45ffba7 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:57.607650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9840c0-e4fc-4fb1-aeae-6b1f60eb06d0 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:57.608638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9840c0-e4fc-4fb1-aeae-6b1f60eb06d0 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:57.609708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9840c0-e4fc-4fb1-aeae-6b1f60eb06d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:57.659912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-1c9840c0-e4fc-4fb1-aeae-6b1f60eb06d0 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:57.661746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1c9840c0-e4fc-4fb1-aeae-6b1f60eb06d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:57.732191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 993/1991] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:57.749673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a86d8-130a-4277-8bc9-7bf72d80f91b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:57.750964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a86d8-130a-4277-8bc9-7bf72d80f91b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:57.750964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a86d8-130a-4277-8bc9-7bf72d80f91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:57.841911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-649a86d8-130a-4277-8bc9-7bf72d80f91b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:57.844301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-649a86d8-130a-4277-8bc9-7bf72d80f91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:57.862206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 999/1992] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:57.862857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9840c0-e4fc-4fb1-aeae-6b1f60eb06d0 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:57.892750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9840c0-e4fc-4fb1-aeae-6b1f60eb06d0 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:57.907348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff825821-bda7-481e-8cc2-f6af7185a271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:57.908056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff825821-bda7-481e-8cc2-f6af7185a271 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:57.909224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff825821-bda7-481e-8cc2-f6af7185a271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:57.909594 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff825821-bda7-481e-8cc2-f6af7185a271 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:57.911123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff825821-bda7-481e-8cc2-f6af7185a271 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:27:57.911449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff825821-bda7-481e-8cc2-f6af7185a271 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:57.913602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff825821-bda7-481e-8cc2-f6af7185a271 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:57.934595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 994/1993] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:27:57.936904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a86d8-130a-4277-8bc9-7bf72d80f91b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:57.947893 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a86d8-130a-4277-8bc9-7bf72d80f91b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:57.953558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63c85c-daa6-4642-829d-af3c5f3dc0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:57.954346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63c85c-daa6-4642-829d-af3c5f3dc0ff admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:57.954723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63c85c-daa6-4642-829d-af3c5f3dc0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:57.955000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63c85c-daa6-4642-829d-af3c5f3dc0ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:57.955749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63c85c-daa6-4642-829d-af3c5f3dc0ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:57.957733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63c85c-daa6-4642-829d-af3c5f3dc0ff admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:58.014857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1000/1994] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:27:57 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:58.017015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff825821-bda7-481e-8cc2-f6af7185a271 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:58.030287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff825821-bda7-481e-8cc2-f6af7185a271 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:58.136346 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:27:58.145913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61ec62-82d5-489f-a036-af94cf08666b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:58.147687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61ec62-82d5-489f-a036-af94cf08666b admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:58.148085 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61ec62-82d5-489f-a036-af94cf08666b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:58.148615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61ec62-82d5-489f-a036-af94cf08666b admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:58.149485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61ec62-82d5-489f-a036-af94cf08666b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:58.151763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61ec62-82d5-489f-a036-af94cf08666b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:58.175935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 995/1995] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:27:58 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:27:58.177224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63c85c-daa6-4642-829d-af3c5f3dc0ff admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:58.189622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63c85c-daa6-4642-829d-af3c5f3dc0ff admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:58.194677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00570fe9-564c-44dc-9c66-b6d8d3567a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:58.195423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00570fe9-564c-44dc-9c66-b6d8d3567a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:58.195784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00570fe9-564c-44dc-9c66-b6d8d3567a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:58.196290 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00570fe9-564c-44dc-9c66-b6d8d3567a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:58.197255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00570fe9-564c-44dc-9c66-b6d8d3567a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:58.199173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00570fe9-564c-44dc-9c66-b6d8d3567a73 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:58.212497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1001/1996] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:58.216334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61ec62-82d5-489f-a036-af94cf08666b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:58.228676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61ec62-82d5-489f-a036-af94cf08666b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:58.235525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665b97c-bd3d-4be0-abe0-12df604252aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:58.236404 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b97c-bd3d-4be0-abe0-12df604252aa admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:58.236680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b97c-bd3d-4be0-abe0-12df604252aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:58.236962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665b97c-bd3d-4be0-abe0-12df604252aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:58.238139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665b97c-bd3d-4be0-abe0-12df604252aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:58.239490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665b97c-bd3d-4be0-abe0-12df604252aa admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:58.255824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 996/1997] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:27:58.257046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00570fe9-564c-44dc-9c66-b6d8d3567a73 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:58.267758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00570fe9-564c-44dc-9c66-b6d8d3567a73 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:58.275169 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9cfe1-92fb-4d60-9f49-5aa4a5cfaede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:58.277179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9cfe1-92fb-4d60-9f49-5aa4a5cfaede admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:58.277576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9cfe1-92fb-4d60-9f49-5aa4a5cfaede admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:58.277968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9cfe1-92fb-4d60-9f49-5aa4a5cfaede admin admin] PATH_INFO: `/v3/projects/98f97ac03be44e0bb8ab21d985d1dd27/users/781d4a4643374bc6a78c3553d62f7672/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:58.301489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9cfe1-92fb-4d60-9f49-5aa4a5cfaede admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f97ac03be44e0bb8ab21d985d1dd27, user_id=781d4a4643374bc6a78c3553d62f7672, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:58.305213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9cfe1-92fb-4d60-9f49-5aa4a5cfaede admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:58.323256 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1002/1998] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:27:58 2022] PUT /identity/v3/projects/98f97ac03be44e0bb8ab21d985d1dd27/users/781d4a4643374bc6a78c3553d62f7672/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:27:58.328452 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd5329-fecb-415f-87e7-fc843db9a087 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:58.329232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd5329-fecb-415f-87e7-fc843db9a087 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:58.329232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd5329-fecb-415f-87e7-fc843db9a087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:58.354187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d4bd5329-fecb-415f-87e7-fc843db9a087 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:27:58.355532 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d4bd5329-fecb-415f-87e7-fc843db9a087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:58.455103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 997/1999] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:27:58.462429 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9cfe1-92fb-4d60-9f49-5aa4a5cfaede admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:27:58.464309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f3f9cfe1-92fb-4d60-9f49-5aa4a5cfaede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:27:58.551418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9cfe1-92fb-4d60-9f49-5aa4a5cfaede admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:27:58.555759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2dce21-6c39-4280-a33a-84b12c64b7a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:27:58.558804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dce21-6c39-4280-a33a-84b12c64b7a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:27:58.559166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dce21-6c39-4280-a33a-84b12c64b7a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:27:58.559509 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dce21-6c39-4280-a33a-84b12c64b7a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:27:58.570138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2dce21-6c39-4280-a33a-84b12c64b7a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:27:58.571890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2dce21-6c39-4280-a33a-84b12c64b7a0 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:27:58.584570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1003/2000] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:28:01.878661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e043aa-1324-4fef-9697-aef19ad8a404 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:01.879179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e043aa-1324-4fef-9697-aef19ad8a404 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:01.879256 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e043aa-1324-4fef-9697-aef19ad8a404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:01.898638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-73e043aa-1324-4fef-9697-aef19ad8a404 None None] MFA Rules not processed for user `dac7f01263374a2cbdd892a2513f5ecd`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:01.899930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-73e043aa-1324-4fef-9697-aef19ad8a404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:01.965862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 998/2001] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:04.736176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33900a64-0827-4f49-b0a8-29e0d14b68b7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:04.737110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33900a64-0827-4f49-b0a8-29e0d14b68b7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:04.737394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33900a64-0827-4f49-b0a8-29e0d14b68b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:04.758941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-33900a64-0827-4f49-b0a8-29e0d14b68b7 None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:04.760253 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-33900a64-0827-4f49-b0a8-29e0d14b68b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:04.785902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29a9f0-7cd0-48de-b0ca-b1357617eeb2 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:04.787435 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29a9f0-7cd0-48de-b0ca-b1357617eeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:04.787938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d29a9f0-7cd0-48de-b0ca-b1357617eeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:04.826144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-4d29a9f0-7cd0-48de-b0ca-b1357617eeb2 None None] MFA Rules not processed for user `781d4a4643374bc6a78c3553d62f7672`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:04.827943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-4d29a9f0-7cd0-48de-b0ca-b1357617eeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:04.845173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1004/2002] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:28:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:04.861559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33900a64-0827-4f49-b0a8-29e0d14b68b7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:04.863763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-33900a64-0827-4f49-b0a8-29e0d14b68b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:04.871911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:04.934623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 999/2003] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:28:04.940521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29a9f0-7cd0-48de-b0ca-b1357617eeb2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:04.956587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d29a9f0-7cd0-48de-b0ca-b1357617eeb2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:04.962302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60322634-9852-4589-93bc-6b090d33b951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:04.963235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60322634-9852-4589-93bc-6b090d33b951 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:04.963517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60322634-9852-4589-93bc-6b090d33b951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:04.963888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60322634-9852-4589-93bc-6b090d33b951 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:04.964770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60322634-9852-4589-93bc-6b090d33b951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:04.967096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60322634-9852-4589-93bc-6b090d33b951 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:04.972250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:04.978310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33900a64-0827-4f49-b0a8-29e0d14b68b7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:04.995195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3c84b-8e19-4ef9-9c16-184a2c91e185 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:04.996096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3c84b-8e19-4ef9-9c16-184a2c91e185 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:04.998961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3c84b-8e19-4ef9-9c16-184a2c91e185 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:04.999161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3c84b-8e19-4ef9-9c16-184a2c91e185 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:05.008229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1000/2004] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:04 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:05.011721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60322634-9852-4589-93bc-6b090d33b951 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:05.019477 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3c84b-8e19-4ef9-9c16-184a2c91e185 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:05.022582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3c84b-8e19-4ef9-9c16-184a2c91e185 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:05.044047 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1005/2005] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:28:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 03 15:28:05.160844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60322634-9852-4589-93bc-6b090d33b951 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:05.177332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696073ef-4090-4403-8dce-d8164e335fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:05.178516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696073ef-4090-4403-8dce-d8164e335fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:05.179277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696073ef-4090-4403-8dce-d8164e335fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:05.179788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696073ef-4090-4403-8dce-d8164e335fbc admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:05.182579 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696073ef-4090-4403-8dce-d8164e335fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:05.200726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696073ef-4090-4403-8dce-d8164e335fbc admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:05.240412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1001/2006] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:05 2022] POST /identity/v3/users => generated 292 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:05.241851 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3c84b-8e19-4ef9-9c16-184a2c91e185 service swift] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:05.266259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3c84b-8e19-4ef9-9c16-184a2c91e185 service swift] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:05.279914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be1bab-db76-4486-8937-a47b983e86c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:05.280885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be1bab-db76-4486-8937-a47b983e86c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:05.281337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be1bab-db76-4486-8937-a47b983e86c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:05.281793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be1bab-db76-4486-8937-a47b983e86c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:05.282708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be1bab-db76-4486-8937-a47b983e86c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:05.284691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be1bab-db76-4486-8937-a47b983e86c1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:05.323591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1006/2007] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:05.328106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696073ef-4090-4403-8dce-d8164e335fbc admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:05.351708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696073ef-4090-4403-8dce-d8164e335fbc admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:05.360550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef64555-9518-4de8-bdd3-a49270183340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:05.362109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64555-9518-4de8-bdd3-a49270183340 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:05.362774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64555-9518-4de8-bdd3-a49270183340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:05.364753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64555-9518-4de8-bdd3-a49270183340 admin admin] PATH_INFO: `/v3/projects/a122683898334dc5b09dc4f24b9370b1/users/e2c24c560e4045a38156ad50757ad7dc/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:05.414167 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef64555-9518-4de8-bdd3-a49270183340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a122683898334dc5b09dc4f24b9370b1, user_id=e2c24c560e4045a38156ad50757ad7dc, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:05.419348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef64555-9518-4de8-bdd3-a49270183340 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:05.462205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1002/2008] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:05 2022] PUT /identity/v3/projects/a122683898334dc5b09dc4f24b9370b1/users/e2c24c560e4045a38156ad50757ad7dc/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:05.463847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be1bab-db76-4486-8937-a47b983e86c1 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:05.477104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be1bab-db76-4486-8937-a47b983e86c1 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:05.720094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dcffb-e347-443e-9310-4ce165f458e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:05.722181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dcffb-e347-443e-9310-4ce165f458e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:05.722799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dcffb-e347-443e-9310-4ce165f458e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:05.724637 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dcffb-e347-443e-9310-4ce165f458e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:05.727715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9dcffb-e347-443e-9310-4ce165f458e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:05.732915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9dcffb-e347-443e-9310-4ce165f458e8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:05.768269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1007/2009] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:05.772050 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef64555-9518-4de8-bdd3-a49270183340 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:05.794265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef64555-9518-4de8-bdd3-a49270183340 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:05.809185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514efa9f-a9ba-482c-a7f5-62c905670df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:05.810120 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514efa9f-a9ba-482c-a7f5-62c905670df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:05.820386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514efa9f-a9ba-482c-a7f5-62c905670df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:05.820386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514efa9f-a9ba-482c-a7f5-62c905670df0 admin admin] PATH_INFO: `/v3/projects/a122683898334dc5b09dc4f24b9370b1/users/e2c24c560e4045a38156ad50757ad7dc/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:05.826721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514efa9f-a9ba-482c-a7f5-62c905670df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a122683898334dc5b09dc4f24b9370b1, user_id=e2c24c560e4045a38156ad50757ad7dc, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:05.829502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514efa9f-a9ba-482c-a7f5-62c905670df0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:05.835019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dcffb-e347-443e-9310-4ce165f458e8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:05.840757 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1c9dcffb-e347-443e-9310-4ce165f458e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:05.872233 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1003/2010] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:05 2022] PUT /identity/v3/projects/a122683898334dc5b09dc4f24b9370b1/users/e2c24c560e4045a38156ad50757ad7dc/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:05.893222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dcffb-e347-443e-9310-4ce165f458e8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:06.057577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:06.059877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:06.060577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:06.061683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] PATH_INFO: `/v3/users/3dc72ca8fa3742e9a7d2e8bae7b992ae` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:06.066366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dc72ca8fa3742e9a7d2e8bae7b992ae)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:06.070693 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:06.130034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:06.176188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dc72ca8fa3742e9a7d2e8bae7b992ae'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:06.192904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dc72ca8fa3742e9a7d2e8bae7b992ae'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:06.194463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dc72ca8fa3742e9a7d2e8bae7b992ae'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:06.203844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dc72ca8fa3742e9a7d2e8bae7b992ae'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:06.205085 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dc72ca8fa3742e9a7d2e8bae7b992ae'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:06.229589 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1008/2011] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:05 2022] DELETE /identity/v3/users/3dc72ca8fa3742e9a7d2e8bae7b992ae => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:06.234037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfd28c-57b0-40a5-bb2d-3b40dd21cfe1 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:06.234686 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfd28c-57b0-40a5-bb2d-3b40dd21cfe1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:06.235118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfd28c-57b0-40a5-bb2d-3b40dd21cfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:06.291786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c1bfd28c-57b0-40a5-bb2d-3b40dd21cfe1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:06.297167 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c1bfd28c-57b0-40a5-bb2d-3b40dd21cfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:06.468996 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1004/2012] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:06.487901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:06.489233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:06.635804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a708b8a-01ae-4196-9688-75950e7ed608 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:06.642454 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d958f0a-4135-4eb3-a0db-24399c2bcba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:06.644174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d958f0a-4135-4eb3-a0db-24399c2bcba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:06.644722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d958f0a-4135-4eb3-a0db-24399c2bcba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:06.645290 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d958f0a-4135-4eb3-a0db-24399c2bcba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:06.670476 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d958f0a-4135-4eb3-a0db-24399c2bcba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:06.672507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d958f0a-4135-4eb3-a0db-24399c2bcba2 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:06.698114 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1009/2013] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:07.657321 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfd28c-57b0-40a5-bb2d-3b40dd21cfe1 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:07.659815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c1bfd28c-57b0-40a5-bb2d-3b40dd21cfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:07.682721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfd28c-57b0-40a5-bb2d-3b40dd21cfe1 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:07.696680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:07.699554 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:07.700151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:07.700722 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] PATH_INFO: `/v3/projects/10c35bb9eda54d509901fab7285c40e0` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:07.702788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10c35bb9eda54d509901fab7285c40e0)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:07.705532 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:07.809053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:07.828262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10c35bb9eda54d509901fab7285c40e0', 'request_id': 'req-0bce3ebd-3288-44d1-ba89-5987191ce793'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:07.840356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10c35bb9eda54d509901fab7285c40e0', 'request_id': 'req-0bce3ebd-3288-44d1-ba89-5987191ce793'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:07.865054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10c35bb9eda54d509901fab7285c40e0', 'request_id': 'req-0bce3ebd-3288-44d1-ba89-5987191ce793'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:07.883491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] The token cache is being invalidate because project 10c35bb9eda54d509901fab7285c40e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:07.884106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:07.884995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-0bce3ebd-3288-44d1-ba89-5987191ce793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:07.893487 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb3373-a93c-499b-ac9f-da4cce14e13e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:07.894504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb3373-a93c-499b-ac9f-da4cce14e13e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:07.895041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb3373-a93c-499b-ac9f-da4cce14e13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:07.897793 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1005/2014] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:07 2022] DELETE /identity/v3/projects/10c35bb9eda54d509901fab7285c40e0 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:07.915172 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-fcfb3373-a93c-499b-ac9f-da4cce14e13e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:07.916915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fcfb3373-a93c-499b-ac9f-da4cce14e13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:08.059814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1010/2015] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:08.068260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e46027-a736-4fb4-9d71-a24bfc89ec3c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:08.068550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e46027-a736-4fb4-9d71-a24bfc89ec3c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:08.068828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e46027-a736-4fb4-9d71-a24bfc89ec3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:08.090428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-f8e46027-a736-4fb4-9d71-a24bfc89ec3c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:08.091679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f8e46027-a736-4fb4-9d71-a24bfc89ec3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:08.100774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1006/2016] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:08.105276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb3373-a93c-499b-ac9f-da4cce14e13e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:08.116535 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb3373-a93c-499b-ac9f-da4cce14e13e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:08.122435 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f2eb1-32ea-4f8a-8548-cff9e3d3ba89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:08.123461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f2eb1-32ea-4f8a-8548-cff9e3d3ba89 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:08.123581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f2eb1-32ea-4f8a-8548-cff9e3d3ba89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:08.124120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f2eb1-32ea-4f8a-8548-cff9e3d3ba89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:08.125194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f2eb1-32ea-4f8a-8548-cff9e3d3ba89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:28:08.125513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f2eb1-32ea-4f8a-8548-cff9e3d3ba89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:08.127609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f2eb1-32ea-4f8a-8548-cff9e3d3ba89 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:08.155720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1011/2017] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:28:08.160727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e46027-a736-4fb4-9d71-a24bfc89ec3c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:08.172532 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e46027-a736-4fb4-9d71-a24bfc89ec3c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:08.178366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92956d00-4b63-44e9-9394-694283f550a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:08.179084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92956d00-4b63-44e9-9394-694283f550a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:08.179370 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92956d00-4b63-44e9-9394-694283f550a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:08.180596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92956d00-4b63-44e9-9394-694283f550a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:08.180596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92956d00-4b63-44e9-9394-694283f550a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:08.182564 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92956d00-4b63-44e9-9394-694283f550a9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:08.214575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1007/2018] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:08 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:08.218671 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f2eb1-32ea-4f8a-8548-cff9e3d3ba89 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:08.238408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f2eb1-32ea-4f8a-8548-cff9e3d3ba89 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:08.290687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:08.381519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74f69b-a7ee-4470-b765-05d790a0de5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:08.382387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74f69b-a7ee-4470-b765-05d790a0de5e admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:08.382765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74f69b-a7ee-4470-b765-05d790a0de5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:08.383058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74f69b-a7ee-4470-b765-05d790a0de5e admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:08.383892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74f69b-a7ee-4470-b765-05d790a0de5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:08.387517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74f69b-a7ee-4470-b765-05d790a0de5e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:08.445372 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1012/2019] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:08 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:08.450238 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92956d00-4b63-44e9-9394-694283f550a9 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:08.467266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92956d00-4b63-44e9-9394-694283f550a9 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:08.493115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2430f4-dc2b-4c21-93c6-64970ddd4289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:08.493801 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2430f4-dc2b-4c21-93c6-64970ddd4289 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:08.494083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2430f4-dc2b-4c21-93c6-64970ddd4289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:08.495136 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2430f4-dc2b-4c21-93c6-64970ddd4289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:08.495136 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2430f4-dc2b-4c21-93c6-64970ddd4289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:08.496870 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2430f4-dc2b-4c21-93c6-64970ddd4289 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:08.532676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1008/2020] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:08.537722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74f69b-a7ee-4470-b765-05d790a0de5e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:08.551348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74f69b-a7ee-4470-b765-05d790a0de5e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:08.559490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b558707-9203-4997-a63f-0feb8e8c0aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:08.560681 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b558707-9203-4997-a63f-0feb8e8c0aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:08.560977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b558707-9203-4997-a63f-0feb8e8c0aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:08.561260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b558707-9203-4997-a63f-0feb8e8c0aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:08.574648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b558707-9203-4997-a63f-0feb8e8c0aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:08.578116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b558707-9203-4997-a63f-0feb8e8c0aa5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:08.600790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1013/2021] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:08.601193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2430f4-dc2b-4c21-93c6-64970ddd4289 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:08.617697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2430f4-dc2b-4c21-93c6-64970ddd4289 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:08.636968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cd7cd-0599-48a1-a3ec-c3f43126da83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:08.637968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cd7cd-0599-48a1-a3ec-c3f43126da83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:08.638258 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cd7cd-0599-48a1-a3ec-c3f43126da83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:08.639348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450cd7cd-0599-48a1-a3ec-c3f43126da83 admin admin] PATH_INFO: `/v3/projects/123c9ca329dd4d2699241875f2720ac3/users/4b4fe74771ee43e7b74a7784dc1cdbe5/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:08.698285 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cd7cd-0599-48a1-a3ec-c3f43126da83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=123c9ca329dd4d2699241875f2720ac3, user_id=4b4fe74771ee43e7b74a7784dc1cdbe5, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:08.707232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450cd7cd-0599-48a1-a3ec-c3f43126da83 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:08.735428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1009/2022] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:08 2022] PUT /identity/v3/projects/123c9ca329dd4d2699241875f2720ac3/users/4b4fe74771ee43e7b74a7784dc1cdbe5/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:28:08.748360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393041b0-0c7f-4b83-8be1-a37d269aae0f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:08.748707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393041b0-0c7f-4b83-8be1-a37d269aae0f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:08.749008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393041b0-0c7f-4b83-8be1-a37d269aae0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:08.794620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-393041b0-0c7f-4b83-8be1-a37d269aae0f None None] MFA Rules not processed for user `4b4fe74771ee43e7b74a7784dc1cdbe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:08.795854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-393041b0-0c7f-4b83-8be1-a37d269aae0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:08.906361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1014/2023] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:08.912604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cd7cd-0599-48a1-a3ec-c3f43126da83 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:08.914349 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-450cd7cd-0599-48a1-a3ec-c3f43126da83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:09.173250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450cd7cd-0599-48a1-a3ec-c3f43126da83 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:09.196730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224cc0c-2e72-485f-bff1-2338301b1b67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:09.199552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224cc0c-2e72-485f-bff1-2338301b1b67 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:09.200643 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224cc0c-2e72-485f-bff1-2338301b1b67 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:09.200948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224cc0c-2e72-485f-bff1-2338301b1b67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:09.246294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224cc0c-2e72-485f-bff1-2338301b1b67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:09.256069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224cc0c-2e72-485f-bff1-2338301b1b67 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:09.308483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1010/2024] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:12.322600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c6b81-6ba9-4030-b213-516462c44ede None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:12.323822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c6b81-6ba9-4030-b213-516462c44ede None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:12.324271 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c6b81-6ba9-4030-b213-516462c44ede None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:12.326043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1015/2025] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:28:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:28:12.397160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155773e-d585-4a54-ae74-72c1c92e5eab None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:12.398227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155773e-d585-4a54-ae74-72c1c92e5eab None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:12.398704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155773e-d585-4a54-ae74-72c1c92e5eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:12.449520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-4155773e-d585-4a54-ae74-72c1c92e5eab None None] MFA Rules not processed for user `50625c2b38ed41698ba6cccdb12730ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:12.450809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-4155773e-d585-4a54-ae74-72c1c92e5eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:12.605432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c6b81-6ba9-4030-b213-516462c44ede None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:12.661056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:12.715785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1011/2026] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:28:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:12.734645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155773e-d585-4a54-ae74-72c1c92e5eab None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:12.736183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-4155773e-d585-4a54-ae74-72c1c92e5eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:12.769143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155773e-d585-4a54-ae74-72c1c92e5eab None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:12.790253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac02b91-7900-4cf3-b25a-4b5763f4ba9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:12.790956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac02b91-7900-4cf3-b25a-4b5763f4ba9d service placement] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:12.791242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac02b91-7900-4cf3-b25a-4b5763f4ba9d service placement] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:12.791519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac02b91-7900-4cf3-b25a-4b5763f4ba9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:12.831021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac02b91-7900-4cf3-b25a-4b5763f4ba9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:12.845469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac02b91-7900-4cf3-b25a-4b5763f4ba9d service placement] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:12.887902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c6b81-6ba9-4030-b213-516462c44ede None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:12.901576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1012/2027] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:12.907194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:12.908097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:12.908373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:12.908643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] PATH_INFO: `/v3/users/4b4fe74771ee43e7b74a7784dc1cdbe5` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:12.912049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b4fe74771ee43e7b74a7784dc1cdbe5)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:12.917426 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:13.038752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:13.043959 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b4fe74771ee43e7b74a7784dc1cdbe5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:13.099083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b4fe74771ee43e7b74a7784dc1cdbe5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:13.100072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b4fe74771ee43e7b74a7784dc1cdbe5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:13.122703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b4fe74771ee43e7b74a7784dc1cdbe5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:13.123458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b4fe74771ee43e7b74a7784dc1cdbe5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:13.163983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1016/2028] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:12 2022] DELETE /identity/v3/users/4b4fe74771ee43e7b74a7784dc1cdbe5 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:28:13.169545 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963a59e-ea91-4882-bd68-e86d47caff81 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:13.169833 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963a59e-ea91-4882-bd68-e86d47caff81 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:13.170290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963a59e-ea91-4882-bd68-e86d47caff81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:13.206333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-9963a59e-ea91-4882-bd68-e86d47caff81 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:13.209148 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9963a59e-ea91-4882-bd68-e86d47caff81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:13.449748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1013/2029] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:13.582806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:13.584524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:13.768564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8051529-dd67-47c9-b667-dae84073b2fc admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:13.782826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519441f-7ca8-4083-98c5-cfea2b8ce5f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:13.783504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519441f-7ca8-4083-98c5-cfea2b8ce5f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:13.783975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519441f-7ca8-4083-98c5-cfea2b8ce5f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:13.784408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519441f-7ca8-4083-98c5-cfea2b8ce5f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:13.813264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519441f-7ca8-4083-98c5-cfea2b8ce5f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:13.815026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519441f-7ca8-4083-98c5-cfea2b8ce5f9 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:13.850824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1017/2030] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:14.080668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963a59e-ea91-4882-bd68-e86d47caff81 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:14.082341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9963a59e-ea91-4882-bd68-e86d47caff81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:14.110798 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963a59e-ea91-4882-bd68-e86d47caff81 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:14.134922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:14.137172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:14.145586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:14.146009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] PATH_INFO: `/v3/users/188d26a3df14484687ff57da147171aa` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:14.152925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=188d26a3df14484687ff57da147171aa)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:14.164377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:14.318775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '188d26a3df14484687ff57da147171aa'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:14.356924 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '188d26a3df14484687ff57da147171aa'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:14.358110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '188d26a3df14484687ff57da147171aa'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:14.377111 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '188d26a3df14484687ff57da147171aa'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:14.378998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '188d26a3df14484687ff57da147171aa'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:14.419665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1014/2031] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:14 2022] DELETE /identity/v3/users/188d26a3df14484687ff57da147171aa => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:14.891330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519441f-7ca8-4083-98c5-cfea2b8ce5f9 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:14.893814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c519441f-7ca8-4083-98c5-cfea2b8ce5f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:14.923868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:14.925488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:14.951524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519441f-7ca8-4083-98c5-cfea2b8ce5f9 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:15.016127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:15.059156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67476643-3189-43d1-b361-1033775b3235 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:15.074066 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:15.079002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:15.081779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:15.082252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] PATH_INFO: `/v3/projects/516fe8f22188479db1b940bc682bf5a0` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:15.086140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] RBAC: Authorizing `identity:delete_project(project_id=516fe8f22188479db1b940bc682bf5a0)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:15.090353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:15.109381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:15.110296 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:15.110625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:15.110912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] PATH_INFO: `/v3/projects/123c9ca329dd4d2699241875f2720ac3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:15.116274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=123c9ca329dd4d2699241875f2720ac3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:15.120753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:15.191532 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '516fe8f22188479db1b940bc682bf5a0', 'request_id': 'req-b1b5752b-d693-4242-b601-5affb0a9f78a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.199321 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '516fe8f22188479db1b940bc682bf5a0', 'request_id': 'req-b1b5752b-d693-4242-b601-5affb0a9f78a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.216923 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '516fe8f22188479db1b940bc682bf5a0', 'request_id': 'req-b1b5752b-d693-4242-b601-5affb0a9f78a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.244689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] The token cache is being invalidate because project 516fe8f22188479db1b940bc682bf5a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:15.245250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.249152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.252630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1015/2032] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:14 2022] DELETE /identity/v3/projects/516fe8f22188479db1b940bc682bf5a0 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:15.256580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:15.260194 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:15.263057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '123c9ca329dd4d2699241875f2720ac3', 'request_id': 'req-30d377af-8ddb-441f-9030-80e6767f66c6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.281791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '123c9ca329dd4d2699241875f2720ac3', 'request_id': 'req-30d377af-8ddb-441f-9030-80e6767f66c6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.287245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:15.301263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '123c9ca329dd4d2699241875f2720ac3', 'request_id': 'req-30d377af-8ddb-441f-9030-80e6767f66c6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.352130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] The token cache is being invalidate because project 123c9ca329dd4d2699241875f2720ac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:15.357057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.357936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-30d377af-8ddb-441f-9030-80e6767f66c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.366957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1018/2033] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:14 2022] DELETE /identity/v3/projects/123c9ca329dd4d2699241875f2720ac3 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:15.373629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57189184-783c-45d8-b13a-90ac042cb700 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:15.373934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57189184-783c-45d8-b13a-90ac042cb700 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:15.374257 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57189184-783c-45d8-b13a-90ac042cb700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:15.419489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-57189184-783c-45d8-b13a-90ac042cb700 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:15.420762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-57189184-783c-45d8-b13a-90ac042cb700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:15.464894 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5752b-d693-4242-b601-5affb0a9f78a admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:15.560009 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1019/2034] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:15.570144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafe40e-67e8-4fc8-a4e2-e45a677f9af6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:15.570554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafe40e-67e8-4fc8-a4e2-e45a677f9af6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:15.571000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafe40e-67e8-4fc8-a4e2-e45a677f9af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:15.596649 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-4bafe40e-67e8-4fc8-a4e2-e45a677f9af6 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:15.598392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4bafe40e-67e8-4fc8-a4e2-e45a677f9af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:15.608900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:15.609865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:15.610636 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:15.611090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] PATH_INFO: `/v3/users/3c8066e46e624db1bed5bade77fed011` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:15.613152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c8066e46e624db1bed5bade77fed011)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:15.615892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:15.620198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1020/2035] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:15.621149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafe40e-67e8-4fc8-a4e2-e45a677f9af6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:15.667428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafe40e-67e8-4fc8-a4e2-e45a677f9af6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:15.684242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a1ddb-2fbe-410c-a834-7f026483bf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:15.685019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a1ddb-2fbe-410c-a834-7f026483bf4f admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:15.685355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a1ddb-2fbe-410c-a834-7f026483bf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:15.685735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a1ddb-2fbe-410c-a834-7f026483bf4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:15.686484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a1ddb-2fbe-410c-a834-7f026483bf4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:28:15.686881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a1ddb-2fbe-410c-a834-7f026483bf4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:15.688442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a1ddb-2fbe-410c-a834-7f026483bf4f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:15.714268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:15.726221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1021/2036] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:28:15.731071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a1ddb-2fbe-410c-a834-7f026483bf4f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:15.753283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a1ddb-2fbe-410c-a834-7f026483bf4f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:15.771315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e65481-8321-4a1c-84ad-9ba78385a400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:15.772272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65481-8321-4a1c-84ad-9ba78385a400 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:15.772621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65481-8321-4a1c-84ad-9ba78385a400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:15.772994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65481-8321-4a1c-84ad-9ba78385a400 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:15.774176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e65481-8321-4a1c-84ad-9ba78385a400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:15.776896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e65481-8321-4a1c-84ad-9ba78385a400 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:15.811406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c8066e46e624db1bed5bade77fed011'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.862024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c8066e46e624db1bed5bade77fed011'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.862938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c8066e46e624db1bed5bade77fed011'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.868903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1022/2037] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:15 2022] POST /identity/v3/projects => generated 355 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:15.869978 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e65481-8321-4a1c-84ad-9ba78385a400 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:15.892514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c8066e46e624db1bed5bade77fed011'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.893414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c8066e46e624db1bed5bade77fed011'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:15.936533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1016/2038] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:15 2022] DELETE /identity/v3/users/3c8066e46e624db1bed5bade77fed011 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:16.112045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e65481-8321-4a1c-84ad-9ba78385a400 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:16.260004 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:16.294381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bcdbd-6c23-48a5-a90a-a652f3e6631a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:16.295263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bcdbd-6c23-48a5-a90a-a652f3e6631a admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:16.295641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bcdbd-6c23-48a5-a90a-a652f3e6631a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:16.296016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480bcdbd-6c23-48a5-a90a-a652f3e6631a admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:16.297136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480bcdbd-6c23-48a5-a90a-a652f3e6631a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:16.300442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480bcdbd-6c23-48a5-a90a-a652f3e6631a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:16.346188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1023/2039] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:15 2022] POST /identity/v3/users => generated 278 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:16.347255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:16.348740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:16.377878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbf88c-4e0d-4ec8-9507-29bbea294c38 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:16.421068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe81073-2e7f-4218-a921-9a64e7ca54b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:16.425036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe81073-2e7f-4218-a921-9a64e7ca54b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:16.425334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe81073-2e7f-4218-a921-9a64e7ca54b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:16.425617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe81073-2e7f-4218-a921-9a64e7ca54b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:16.426383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe81073-2e7f-4218-a921-9a64e7ca54b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:16.428260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe81073-2e7f-4218-a921-9a64e7ca54b3 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:16.469020 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1017/2040] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:16.470289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bcdbd-6c23-48a5-a90a-a652f3e6631a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:16.482328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480bcdbd-6c23-48a5-a90a-a652f3e6631a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:16.489113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c4f23-4e00-48d2-857e-5c801f511ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:16.489896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c4f23-4e00-48d2-857e-5c801f511ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:16.490170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c4f23-4e00-48d2-857e-5c801f511ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:16.490469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c4f23-4e00-48d2-857e-5c801f511ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:16.497522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c4f23-4e00-48d2-857e-5c801f511ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:16.499327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c4f23-4e00-48d2-857e-5c801f511ca3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:16.512257 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1024/2041] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:16.517227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe81073-2e7f-4218-a921-9a64e7ca54b3 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:16.551117 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe81073-2e7f-4218-a921-9a64e7ca54b3 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:16.571101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95b741-a635-4dd9-b77b-657bc0dfde4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:16.574833 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95b741-a635-4dd9-b77b-657bc0dfde4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:16.575065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95b741-a635-4dd9-b77b-657bc0dfde4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:16.580653 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95b741-a635-4dd9-b77b-657bc0dfde4b admin admin] PATH_INFO: `/v3/projects/ddbe35f7af89474d850b2c40c8cfe89a/users/b86206ccd3864f7697ac51678624c7ec/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:16.674800 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95b741-a635-4dd9-b77b-657bc0dfde4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbe35f7af89474d850b2c40c8cfe89a, user_id=b86206ccd3864f7697ac51678624c7ec, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:16.690247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95b741-a635-4dd9-b77b-657bc0dfde4b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:16.765904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca72638-2f1a-4907-93ab-5a7a0a43e30b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:16.766376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca72638-2f1a-4907-93ab-5a7a0a43e30b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:16.766916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca72638-2f1a-4907-93ab-5a7a0a43e30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:16.767912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1018/2042] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:16 2022] PUT /identity/v3/projects/ddbe35f7af89474d850b2c40c8cfe89a/users/b86206ccd3864f7697ac51678624c7ec/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:16.841594 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7ca72638-2f1a-4907-93ab-5a7a0a43e30b None None] MFA Rules not processed for user `b86206ccd3864f7697ac51678624c7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:16.843332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7ca72638-2f1a-4907-93ab-5a7a0a43e30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:16.930017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95b741-a635-4dd9-b77b-657bc0dfde4b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:16.933083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1025/2043] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:16 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:17.040769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca72638-2f1a-4907-93ab-5a7a0a43e30b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:17.042232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7ca72638-2f1a-4907-93ab-5a7a0a43e30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:17.122422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95b741-a635-4dd9-b77b-657bc0dfde4b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:17.143278 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1575f371-1e05-4ee1-9c70-b99487bd970a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:17.143982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f371-1e05-4ee1-9c70-b99487bd970a admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:17.144277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f371-1e05-4ee1-9c70-b99487bd970a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:17.144553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1575f371-1e05-4ee1-9c70-b99487bd970a admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:17.145473 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575f371-1e05-4ee1-9c70-b99487bd970a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:17.152457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1575f371-1e05-4ee1-9c70-b99487bd970a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:17.222098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1019/2044] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:16 2022] POST /identity/v3/projects => generated 355 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:17.223747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e0eb6-1c92-4bf1-b0bc-f1565df159ee None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:17.224023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e0eb6-1c92-4bf1-b0bc-f1565df159ee None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:17.224307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e0eb6-1c92-4bf1-b0bc-f1565df159ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:17.259976 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca72638-2f1a-4907-93ab-5a7a0a43e30b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:17.308472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-031e0eb6-1c92-4bf1-b0bc-f1565df159ee None None] MFA Rules not processed for user `e2c24c560e4045a38156ad50757ad7dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:17.309732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-031e0eb6-1c92-4bf1-b0bc-f1565df159ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:17.414346 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:17.425168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:17.427835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:17.428153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:17.428608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] PATH_INFO: `/v3/projects/20293df30c384ad9a8b421f89d79c920` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:17.433154 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=20293df30c384ad9a8b421f89d79c920)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:17.435356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:17.469126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:17.586131 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20293df30c384ad9a8b421f89d79c920', 'request_id': 'req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:17.599876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1020/2045] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:17 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:17.602878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e0eb6-1c92-4bf1-b0bc-f1565df159ee None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:17.603584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20293df30c384ad9a8b421f89d79c920', 'request_id': 'req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:17.623320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e0eb6-1c92-4bf1-b0bc-f1565df159ee None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:17.632691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20293df30c384ad9a8b421f89d79c920', 'request_id': 'req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:17.679505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] The token cache is being invalidate because project 20293df30c384ad9a8b421f89d79c920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:17.681582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:17.683008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:17.685054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1026/2046] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:17 2022] DELETE /identity/v3/projects/20293df30c384ad9a8b421f89d79c920 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:17.688428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:17.694774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:17.802625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e65123-bbb4-477f-b63e-2a1c23549f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:17.802625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65123-bbb4-477f-b63e-2a1c23549f8b admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:17.802992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65123-bbb4-477f-b63e-2a1c23549f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:17.803129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e65123-bbb4-477f-b63e-2a1c23549f8b admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:17.804057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e65123-bbb4-477f-b63e-2a1c23549f8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:17.810971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e65123-bbb4-477f-b63e-2a1c23549f8b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:17.870782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1021/2047] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:17 2022] POST /identity/v3/users => generated 284 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:17.872537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01cfce-52eb-4cb0-b706-78ecdfbdc74e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:17.873029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01cfce-52eb-4cb0-b706-78ecdfbdc74e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:17.873457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01cfce-52eb-4cb0-b706-78ecdfbdc74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:17.906589 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839e58c-cef3-481a-a4cf-a49cfee1ce2f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:17.915671 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a24e0-3ab0-4231-abf0-21ab72fe0df6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:17.915968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a24e0-3ab0-4231-abf0-21ab72fe0df6 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:17.916255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a24e0-3ab0-4231-abf0-21ab72fe0df6 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:17.916530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a24e0-3ab0-4231-abf0-21ab72fe0df6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:17.918786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9e1a24e0-3ab0-4231-abf0-21ab72fe0df6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:17.934065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-5f01cfce-52eb-4cb0-b706-78ecdfbdc74e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:17.935269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5f01cfce-52eb-4cb0-b706-78ecdfbdc74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:17.957327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1022/2048] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:17.965220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01cfce-52eb-4cb0-b706-78ecdfbdc74e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:17.970285 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5f01cfce-52eb-4cb0-b706-78ecdfbdc74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:17.989202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a24e0-3ab0-4231-abf0-21ab72fe0df6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:17.990888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a24e0-3ab0-4231-abf0-21ab72fe0df6 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:17.993929 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f01cfce-52eb-4cb0-b706-78ecdfbdc74e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:18.000679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed30e2-d12f-461a-9010-313d9fd71341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:18.001363 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed30e2-d12f-461a-9010-313d9fd71341 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.001645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed30e2-d12f-461a-9010-313d9fd71341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.001758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1027/2049] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:18.001921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed30e2-d12f-461a-9010-313d9fd71341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.002697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed30e2-d12f-461a-9010-313d9fd71341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:18.003388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56105-8fd3-4e35-a23c-5b2b6707d27d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.003905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56105-8fd3-4e35-a23c-5b2b6707d27d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.004386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be56105-8fd3-4e35-a23c-5b2b6707d27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.004501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ed30e2-d12f-461a-9010-313d9fd71341 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:18.016259 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1023/2050] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:18.020876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed30e2-d12f-461a-9010-313d9fd71341 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:18.028892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2be56105-8fd3-4e35-a23c-5b2b6707d27d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:18.030474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2be56105-8fd3-4e35-a23c-5b2b6707d27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:18.031336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed30e2-d12f-461a-9010-313d9fd71341 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:18.036756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a86a13-bd1c-4b0a-94e4-f52e1d5d723a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:18.037775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a86a13-bd1c-4b0a-94e4-f52e1d5d723a admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.038129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a86a13-bd1c-4b0a-94e4-f52e1d5d723a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.038477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a86a13-bd1c-4b0a-94e4-f52e1d5d723a admin admin] PATH_INFO: `/v3/projects/71b097645e824220be4d02f95c561827/users/bc7753395a56447b8d90acb42d85f258/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.041311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1028/2051] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:18.047979 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be56105-8fd3-4e35-a23c-5b2b6707d27d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:18.064028 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be56105-8fd3-4e35-a23c-5b2b6707d27d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:18.066264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a86a13-bd1c-4b0a-94e4-f52e1d5d723a admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b097645e824220be4d02f95c561827, user_id=bc7753395a56447b8d90acb42d85f258, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:18.069398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66454f5-a968-45c1-9518-38b6beae5afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:18.070073 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66454f5-a968-45c1-9518-38b6beae5afc admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.070462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66454f5-a968-45c1-9518-38b6beae5afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.070772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66454f5-a968-45c1-9518-38b6beae5afc admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.071376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a86a13-bd1c-4b0a-94e4-f52e1d5d723a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:18.071807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66454f5-a968-45c1-9518-38b6beae5afc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:28:18.072524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66454f5-a968-45c1-9518-38b6beae5afc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:18.074169 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66454f5-a968-45c1-9518-38b6beae5afc admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:18.094250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1029/2052] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:28:18.095446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66454f5-a968-45c1-9518-38b6beae5afc admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:18.104005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1024/2053] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:18 2022] PUT /identity/v3/projects/71b097645e824220be4d02f95c561827/users/bc7753395a56447b8d90acb42d85f258/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:28:18.106134 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a86a13-bd1c-4b0a-94e4-f52e1d5d723a admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:18.110717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66454f5-a968-45c1-9518-38b6beae5afc admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:18.138704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:18.222671 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9074a-2ada-44e6-968f-7300bcf2eb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:18.224423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9074a-2ada-44e6-968f-7300bcf2eb2d admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.224852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9074a-2ada-44e6-968f-7300bcf2eb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.225311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9074a-2ada-44e6-968f-7300bcf2eb2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.225981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a86a13-bd1c-4b0a-94e4-f52e1d5d723a admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:18.227035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9074a-2ada-44e6-968f-7300bcf2eb2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:18.230035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9074a-2ada-44e6-968f-7300bcf2eb2d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:18.232053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc288e-f2ac-4b1d-8493-607816542800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:18.232739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc288e-f2ac-4b1d-8493-607816542800 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.233214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc288e-f2ac-4b1d-8493-607816542800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.233604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc288e-f2ac-4b1d-8493-607816542800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.234679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc288e-f2ac-4b1d-8493-607816542800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:18.237087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc288e-f2ac-4b1d-8493-607816542800 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:18.250179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1025/2054] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:18.255811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc288e-f2ac-4b1d-8493-607816542800 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:18.281404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1030/2055] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:18 2022] POST /identity/v3/projects => generated 375 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:18.282786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc288e-f2ac-4b1d-8493-607816542800 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:18.282929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9074a-2ada-44e6-968f-7300bcf2eb2d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:18.302494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:18.380391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7300c54-11f4-4b4d-96fd-5d499a30a30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:18.381485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7300c54-11f4-4b4d-96fd-5d499a30a30a admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.382988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9074a-2ada-44e6-968f-7300bcf2eb2d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:18.387766 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7300c54-11f4-4b4d-96fd-5d499a30a30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.388169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7300c54-11f4-4b4d-96fd-5d499a30a30a admin admin] PATH_INFO: `/v3/projects/71b097645e824220be4d02f95c561827/users/bc7753395a56447b8d90acb42d85f258/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.391047 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd144611-9091-4179-b3a7-edfa2504c5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:18.392036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd144611-9091-4179-b3a7-edfa2504c5bb admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.392285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd144611-9091-4179-b3a7-edfa2504c5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.392563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd144611-9091-4179-b3a7-edfa2504c5bb admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.396528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd144611-9091-4179-b3a7-edfa2504c5bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:18.396628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7300c54-11f4-4b4d-96fd-5d499a30a30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b097645e824220be4d02f95c561827, user_id=bc7753395a56447b8d90acb42d85f258, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:18.396721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd144611-9091-4179-b3a7-edfa2504c5bb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:18.401432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7300c54-11f4-4b4d-96fd-5d499a30a30a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:18.418067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1026/2056] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:18 2022] PUT /identity/v3/projects/71b097645e824220be4d02f95c561827/users/bc7753395a56447b8d90acb42d85f258/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:18.434007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84d54-ba20-49c7-b787-a3ca9411234f None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.434964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84d54-ba20-49c7-b787-a3ca9411234f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.435465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84d54-ba20-49c7-b787-a3ca9411234f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.445096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1031/2057] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:18 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:18.446260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd144611-9091-4179-b3a7-edfa2504c5bb admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:18.458714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd144611-9091-4179-b3a7-edfa2504c5bb admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:18.471024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-68b84d54-ba20-49c7-b787-a3ca9411234f None None] MFA Rules not processed for user `bc7753395a56447b8d90acb42d85f258`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:18.472780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-68b84d54-ba20-49c7-b787-a3ca9411234f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:18.573145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7ad91-f447-46a9-9df5-ac3bc109119d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:18.576654 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7ad91-f447-46a9-9df5-ac3bc109119d admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.578833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7ad91-f447-46a9-9df5-ac3bc109119d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.579132 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7ad91-f447-46a9-9df5-ac3bc109119d admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.580091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7ad91-f447-46a9-9df5-ac3bc109119d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:18.584034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7ad91-f447-46a9-9df5-ac3bc109119d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:18.602172 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1032/2058] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:18.605124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1027/2059] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:18.606103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7ad91-f447-46a9-9df5-ac3bc109119d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:18.617516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b84d54-ba20-49c7-b787-a3ca9411234f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:18.619762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-68b84d54-ba20-49c7-b787-a3ca9411234f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:18.628149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7ad91-f447-46a9-9df5-ac3bc109119d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:18.642922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fcf9e-34c0-49c7-9105-5c44af954e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:18.647153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fcf9e-34c0-49c7-9105-5c44af954e36 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.647153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fcf9e-34c0-49c7-9105-5c44af954e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.647153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fcf9e-34c0-49c7-9105-5c44af954e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.647978 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fcf9e-34c0-49c7-9105-5c44af954e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:18.657337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fcf9e-34c0-49c7-9105-5c44af954e36 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:18.684951 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1033/2060] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:18.685957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fcf9e-34c0-49c7-9105-5c44af954e36 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:18.699298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fcf9e-34c0-49c7-9105-5c44af954e36 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:18.706612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e653ed-4af3-49db-a783-367ad91258b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:18.709439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e653ed-4af3-49db-a783-367ad91258b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.710288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e653ed-4af3-49db-a783-367ad91258b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.710594 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e653ed-4af3-49db-a783-367ad91258b2 admin admin] PATH_INFO: `/v3/projects/3ceaeab3047d416ea09b0717930d7880/users/05b3aed844084eb5968041b0a03f6c70/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.724526 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b84d54-ba20-49c7-b787-a3ca9411234f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:18.729298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890ef8aa-42e7-4ae8-934b-d27cb74d0d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:18.730318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ef8aa-42e7-4ae8-934b-d27cb74d0d1f service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.730801 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ef8aa-42e7-4ae8-934b-d27cb74d0d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.731236 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890ef8aa-42e7-4ae8-934b-d27cb74d0d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.734445 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:18.745732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ef8aa-42e7-4ae8-934b-d27cb74d0d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:18.746099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e653ed-4af3-49db-a783-367ad91258b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ceaeab3047d416ea09b0717930d7880, user_id=05b3aed844084eb5968041b0a03f6c70, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:18.747936 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890ef8aa-42e7-4ae8-934b-d27cb74d0d1f service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:18.751924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e653ed-4af3-49db-a783-367ad91258b2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:18.771410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1028/2061] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:18 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:28:18.777577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1034/2062] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:18 2022] PUT /identity/v3/projects/3ceaeab3047d416ea09b0717930d7880/users/05b3aed844084eb5968041b0a03f6c70/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:18.790096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c912519-30e4-4375-b070-b1d1c3288433 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:18.790381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c912519-30e4-4375-b070-b1d1c3288433 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:18.790729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c912519-30e4-4375-b070-b1d1c3288433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:18.845949 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-9c912519-30e4-4375-b070-b1d1c3288433 None None] MFA Rules not processed for user `05b3aed844084eb5968041b0a03f6c70`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:18.849791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9c912519-30e4-4375-b070-b1d1c3288433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:18.907161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e653ed-4af3-49db-a783-367ad91258b2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:18.931102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:19.023164 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e653ed-4af3-49db-a783-367ad91258b2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:19.029135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948e0eb-4825-45b6-b9bd-825d0a9c07b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:19.030256 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948e0eb-4825-45b6-b9bd-825d0a9c07b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:19.030790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948e0eb-4825-45b6-b9bd-825d0a9c07b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:19.031330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948e0eb-4825-45b6-b9bd-825d0a9c07b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:19.033271 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6948e0eb-4825-45b6-b9bd-825d0a9c07b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:19.055018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1029/2063] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:19.056148 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c912519-30e4-4375-b070-b1d1c3288433 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:19.058226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9c912519-30e4-4375-b070-b1d1c3288433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:19.075120 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c912519-30e4-4375-b070-b1d1c3288433 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:19.086090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb293d-8fe2-4564-8201-f7523af7bd9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:19.087816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb293d-8fe2-4564-8201-f7523af7bd9e service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:19.090264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb293d-8fe2-4564-8201-f7523af7bd9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:19.090732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb293d-8fe2-4564-8201-f7523af7bd9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:19.116528 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb293d-8fe2-4564-8201-f7523af7bd9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:19.118251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb293d-8fe2-4564-8201-f7523af7bd9e service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:19.139622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1030/2064] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:19.185340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948e0eb-4825-45b6-b9bd-825d0a9c07b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:19.187233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948e0eb-4825-45b6-b9bd-825d0a9c07b2 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:19.208981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1035/2065] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:18 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:19.968875 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06a1e1-66e3-4d09-a30b-358c8bf55f1c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:19.969478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06a1e1-66e3-4d09-a30b-358c8bf55f1c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:19.969478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06a1e1-66e3-4d09-a30b-358c8bf55f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:19.988061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-bb06a1e1-66e3-4d09-a30b-358c8bf55f1c None None] MFA Rules not processed for user `781d4a4643374bc6a78c3553d62f7672`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:19.989315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-bb06a1e1-66e3-4d09-a30b-358c8bf55f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:20.052137 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1031/2066] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:29.201366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=51,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:30.044588 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=46 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:40.746191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948e0eb-4825-45b6-b9bd-825d0a9c07b2 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:40.864781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948e0eb-4825-45b6-b9bd-825d0a9c07b2 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:40.869376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:40.870235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:40.870514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:40.870824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] PATH_INFO: `/v3/users/05b3aed844084eb5968041b0a03f6c70` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:40.872010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b3aed844084eb5968041b0a03f6c70)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:40.873930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:40.919993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b3aed844084eb5968041b0a03f6c70'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:40.933146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b3aed844084eb5968041b0a03f6c70'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:40.933860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b3aed844084eb5968041b0a03f6c70'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:40.940792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b3aed844084eb5968041b0a03f6c70'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:40.941587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b3aed844084eb5968041b0a03f6c70'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:40.954981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1036/2067] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:40 2022] DELETE /identity/v3/users/05b3aed844084eb5968041b0a03f6c70 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:40.960214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8329d-3086-4bcd-ad8b-bc6964a5210e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:40.960796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8329d-3086-4bcd-ad8b-bc6964a5210e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:40.960796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa8329d-3086-4bcd-ad8b-bc6964a5210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:41.025070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-5fa8329d-3086-4bcd-ad8b-bc6964a5210e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:41.026384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5fa8329d-3086-4bcd-ad8b-bc6964a5210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:41.089335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1032/2068] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:41.104712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:41.104712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:41.148732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:41.205867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428cd1d-8b69-45a7-a13a-a9f1cfabc68c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:41.210745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b211d27-6a43-481f-bf17-420386e6e5b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:41.211654 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211d27-6a43-481f-bf17-420386e6e5b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:41.211876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211d27-6a43-481f-bf17-420386e6e5b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:41.212179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b211d27-6a43-481f-bf17-420386e6e5b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:41.222990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b211d27-6a43-481f-bf17-420386e6e5b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:41.224473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b211d27-6a43-481f-bf17-420386e6e5b9 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:41.234314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1037/2069] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:41.295149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8329d-3086-4bcd-ad8b-bc6964a5210e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:41.296680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5fa8329d-3086-4bcd-ad8b-bc6964a5210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:41.310841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa8329d-3086-4bcd-ad8b-bc6964a5210e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:41.316894 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:41.317973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:41.318298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:41.318602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] PATH_INFO: `/v3/users/b86206ccd3864f7697ac51678624c7ec` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:41.320152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86206ccd3864f7697ac51678624c7ec)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:41.322482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:41.374375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86206ccd3864f7697ac51678624c7ec'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:41.386622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86206ccd3864f7697ac51678624c7ec'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:41.387351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86206ccd3864f7697ac51678624c7ec'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:41.396368 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86206ccd3864f7697ac51678624c7ec'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:41.396996 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86206ccd3864f7697ac51678624c7ec'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:41.417227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1033/2070] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:41 2022] DELETE /identity/v3/users/b86206ccd3864f7697ac51678624c7ec => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:41.421490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e333fd-860e-4f0c-97ab-63660dec7960 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:41.421783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e333fd-860e-4f0c-97ab-63660dec7960 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:41.422091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e333fd-860e-4f0c-97ab-63660dec7960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:41.440461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-24e333fd-860e-4f0c-97ab-63660dec7960 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:41.442199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-24e333fd-860e-4f0c-97ab-63660dec7960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:41.506319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1038/2071] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:41.517040 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:41.518635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:41.525538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e333fd-860e-4f0c-97ab-63660dec7960 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:41.527428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-24e333fd-860e-4f0c-97ab-63660dec7960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:41.542630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e333fd-860e-4f0c-97ab-63660dec7960 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:41.547641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:41.548525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:41.548805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:41.549080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] PATH_INFO: `/v3/projects/3ceaeab3047d416ea09b0717930d7880` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:41.550254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ceaeab3047d416ea09b0717930d7880)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:41.552267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:41.596249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:41.602672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c2a0f-aec5-4742-b39e-6de14f4e77a0 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:41.603605 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ceaeab3047d416ea09b0717930d7880', 'request_id': 'req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:41.610127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ceaeab3047d416ea09b0717930d7880', 'request_id': 'req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:41.617959 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ceaeab3047d416ea09b0717930d7880', 'request_id': 'req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:41.630873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] The token cache is being invalidate because project 3ceaeab3047d416ea09b0717930d7880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:41.631160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:41.631883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-dd65c700-0859-4ac9-8c28-c5f83c48bdeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:41.633452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1039/2072] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:41 2022] DELETE /identity/v3/projects/3ceaeab3047d416ea09b0717930d7880 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:28:41.638163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933d2fc-a158-4fdc-91df-3a7520f5f7a1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:41.638447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933d2fc-a158-4fdc-91df-3a7520f5f7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:41.638751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933d2fc-a158-4fdc-91df-3a7520f5f7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:41.656576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab851ed6-5909-4672-b320-a25c319f7bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:41.657351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab851ed6-5909-4672-b320-a25c319f7bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:41.657631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab851ed6-5909-4672-b320-a25c319f7bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:41.657913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab851ed6-5909-4672-b320-a25c319f7bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:41.659926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ab851ed6-5909-4672-b320-a25c319f7bf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:41.680508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f933d2fc-a158-4fdc-91df-3a7520f5f7a1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:41.681846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f933d2fc-a158-4fdc-91df-3a7520f5f7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:41.693088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:41.734817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab851ed6-5909-4672-b320-a25c319f7bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:41.736230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab851ed6-5909-4672-b320-a25c319f7bf4 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:41.749886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1034/2073] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:41.758892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1040/2074] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:41.766393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf229a0d-0adf-4073-861e-88e5bfbaa1d7 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:41.766717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf229a0d-0adf-4073-861e-88e5bfbaa1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:41.766997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf229a0d-0adf-4073-861e-88e5bfbaa1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:41.786810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-cf229a0d-0adf-4073-861e-88e5bfbaa1d7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:41.788275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-cf229a0d-0adf-4073-861e-88e5bfbaa1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:41.798245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1035/2075] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:28:41.802924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933d2fc-a158-4fdc-91df-3a7520f5f7a1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:41.815559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933d2fc-a158-4fdc-91df-3a7520f5f7a1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:41.822511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d38a58-aa78-4fb7-b235-b44cfbe8d375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:41.823301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d38a58-aa78-4fb7-b235-b44cfbe8d375 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:41.823735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d38a58-aa78-4fb7-b235-b44cfbe8d375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:41.824111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d38a58-aa78-4fb7-b235-b44cfbe8d375 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:41.825436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d38a58-aa78-4fb7-b235-b44cfbe8d375 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:28:41.825787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d38a58-aa78-4fb7-b235-b44cfbe8d375 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:41.827483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d38a58-aa78-4fb7-b235-b44cfbe8d375 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:41.845307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1041/2076] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:28:41.848270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf229a0d-0adf-4073-861e-88e5bfbaa1d7 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:41.858717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf229a0d-0adf-4073-861e-88e5bfbaa1d7 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:41.865053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e93694-c8cd-4cab-b67b-5b5d6a5b3bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:41.865975 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e93694-c8cd-4cab-b67b-5b5d6a5b3bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:41.866413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e93694-c8cd-4cab-b67b-5b5d6a5b3bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:41.866783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e93694-c8cd-4cab-b67b-5b5d6a5b3bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:41.867757 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e93694-c8cd-4cab-b67b-5b5d6a5b3bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:41.869862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e93694-c8cd-4cab-b67b-5b5d6a5b3bf8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:41.922610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1036/2077] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:41 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:41.924218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d38a58-aa78-4fb7-b235-b44cfbe8d375 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:41.935040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d38a58-aa78-4fb7-b235-b44cfbe8d375 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:41.997962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdd5ac-3926-4a96-b51c-2e5c11883147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:41.998685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdd5ac-3926-4a96-b51c-2e5c11883147 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:41.998972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdd5ac-3926-4a96-b51c-2e5c11883147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:41.999250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdd5ac-3926-4a96-b51c-2e5c11883147 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:41.999963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdd5ac-3926-4a96-b51c-2e5c11883147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.002107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdd5ac-3926-4a96-b51c-2e5c11883147 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.026422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1042/2078] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:41 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:42.030484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e93694-c8cd-4cab-b67b-5b5d6a5b3bf8 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.042785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e93694-c8cd-4cab-b67b-5b5d6a5b3bf8 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.047624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56369f5-e7f5-4001-8dca-c25ba7a981bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.048289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56369f5-e7f5-4001-8dca-c25ba7a981bd admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.048574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56369f5-e7f5-4001-8dca-c25ba7a981bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.048853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56369f5-e7f5-4001-8dca-c25ba7a981bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.049621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56369f5-e7f5-4001-8dca-c25ba7a981bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.051283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56369f5-e7f5-4001-8dca-c25ba7a981bd admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.061788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1037/2079] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:42.065783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdd5ac-3926-4a96-b51c-2e5c11883147 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.076461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdd5ac-3926-4a96-b51c-2e5c11883147 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.081045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c2e74-f9ad-4544-be8d-b4630bacee29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.082196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c2e74-f9ad-4544-be8d-b4630bacee29 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.082475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c2e74-f9ad-4544-be8d-b4630bacee29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.082794 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c2e74-f9ad-4544-be8d-b4630bacee29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.083512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c2e74-f9ad-4544-be8d-b4630bacee29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.085312 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c2e74-f9ad-4544-be8d-b4630bacee29 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.103809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1043/2080] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:42.108783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56369f5-e7f5-4001-8dca-c25ba7a981bd admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.116134 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c2e74-f9ad-4544-be8d-b4630bacee29 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.119230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e52c2e74-f9ad-4544-be8d-b4630bacee29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:42.121303 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56369f5-e7f5-4001-8dca-c25ba7a981bd admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.126381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bc24d-563e-4380-a6f7-0b0ec1899f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.127305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bc24d-563e-4380-a6f7-0b0ec1899f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.127589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bc24d-563e-4380-a6f7-0b0ec1899f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.136315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c2e74-f9ad-4544-be8d-b4630bacee29 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.139008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bc24d-563e-4380-a6f7-0b0ec1899f6e admin admin] PATH_INFO: `/v3/projects/eed335082a544fb88ece841ae1035aac/users/1f607d90947a49a3ae38c4e8de636474/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.143467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.144884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.145166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.145701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] PATH_INFO: `/v3/projects/ddbe35f7af89474d850b2c40c8cfe89a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.146998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddbe35f7af89474d850b2c40c8cfe89a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.149479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.158478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bc24d-563e-4380-a6f7-0b0ec1899f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed335082a544fb88ece841ae1035aac, user_id=1f607d90947a49a3ae38c4e8de636474, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.162464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bc24d-563e-4380-a6f7-0b0ec1899f6e admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.177773 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1038/2081] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:42 2022] PUT /identity/v3/projects/eed335082a544fb88ece841ae1035aac/users/1f607d90947a49a3ae38c4e8de636474/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:42.189034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760192a-05e5-459a-8392-0bb88a85979e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.189433 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760192a-05e5-459a-8392-0bb88a85979e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.189777 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760192a-05e5-459a-8392-0bb88a85979e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.199826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddbe35f7af89474d850b2c40c8cfe89a', 'request_id': 'req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.205393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddbe35f7af89474d850b2c40c8cfe89a', 'request_id': 'req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.215065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddbe35f7af89474d850b2c40c8cfe89a', 'request_id': 'req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.218430 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0760192a-05e5-459a-8392-0bb88a85979e None None] MFA Rules not processed for user `1f607d90947a49a3ae38c4e8de636474`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:42.219737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0760192a-05e5-459a-8392-0bb88a85979e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:42.227346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] The token cache is being invalidate because project ddbe35f7af89474d850b2c40c8cfe89a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:42.227631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.228343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.230001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1044/2082] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:42 2022] DELETE /identity/v3/projects/ddbe35f7af89474d850b2c40c8cfe89a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:28:42.233764 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.235133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:42.241454 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:42.260087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:42.288740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1039/2083] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:28:42.293095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760192a-05e5-459a-8392-0bb88a85979e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.294382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0760192a-05e5-459a-8392-0bb88a85979e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:42.303238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbf6a3-54e7-4e8b-8384-b17a4d70f3fa admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.307908 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760192a-05e5-459a-8392-0bb88a85979e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.308553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.309447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.309714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.309984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] PATH_INFO: `/v3/users/bc7753395a56447b8d90acb42d85f258` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.311213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7753395a56447b8d90acb42d85f258)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.313017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d60da1-75a8-463b-abb1-eb4c0b6e8978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.313142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.313667 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d60da1-75a8-463b-abb1-eb4c0b6e8978 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.313948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d60da1-75a8-463b-abb1-eb4c0b6e8978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.314222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d60da1-75a8-463b-abb1-eb4c0b6e8978 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.315044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d60da1-75a8-463b-abb1-eb4c0b6e8978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.317607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d60da1-75a8-463b-abb1-eb4c0b6e8978 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.347288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1040/2084] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:42 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:42.351690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d60da1-75a8-463b-abb1-eb4c0b6e8978 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.368421 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7753395a56447b8d90acb42d85f258'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.383392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7753395a56447b8d90acb42d85f258'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.384156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7753395a56447b8d90acb42d85f258'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.390894 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7753395a56447b8d90acb42d85f258'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.391545 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7753395a56447b8d90acb42d85f258'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.406149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1045/2085] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:42 2022] DELETE /identity/v3/users/bc7753395a56447b8d90acb42d85f258 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:28:42.416165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d60da1-75a8-463b-abb1-eb4c0b6e8978 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.477086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5639d3-59f4-4e46-8a02-9b439d2de03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.477832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5639d3-59f4-4e46-8a02-9b439d2de03c admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.478166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5639d3-59f4-4e46-8a02-9b439d2de03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.478442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5639d3-59f4-4e46-8a02-9b439d2de03c admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.479190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5639d3-59f4-4e46-8a02-9b439d2de03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.481264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5639d3-59f4-4e46-8a02-9b439d2de03c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.502013 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1041/2086] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:42 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:42.506428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.508554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:42.522574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10dbb9-2e74-4a83-aada-1c76ab129284 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.527228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c3260-e548-42d4-b369-02684259ccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.527895 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c3260-e548-42d4-b369-02684259ccf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.528177 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c3260-e548-42d4-b369-02684259ccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.528450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c3260-e548-42d4-b369-02684259ccf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.529176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c3260-e548-42d4-b369-02684259ccf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.530819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c3260-e548-42d4-b369-02684259ccf4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.541381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1046/2087] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:42.545458 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5639d3-59f4-4e46-8a02-9b439d2de03c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.562494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5639d3-59f4-4e46-8a02-9b439d2de03c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.567995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfa0d0-b754-43ce-97e3-e09d9885fa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.568714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfa0d0-b754-43ce-97e3-e09d9885fa42 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.569119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfa0d0-b754-43ce-97e3-e09d9885fa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.569485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfa0d0-b754-43ce-97e3-e09d9885fa42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.570215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfa0d0-b754-43ce-97e3-e09d9885fa42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.571992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfa0d0-b754-43ce-97e3-e09d9885fa42 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.582415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1042/2088] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:42.586613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c3260-e548-42d4-b369-02684259ccf4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.596665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c3260-e548-42d4-b369-02684259ccf4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.601289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe0436-7112-4f96-a26e-a1821baac13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.602177 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe0436-7112-4f96-a26e-a1821baac13b admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.602454 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe0436-7112-4f96-a26e-a1821baac13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.602755 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe0436-7112-4f96-a26e-a1821baac13b admin admin] PATH_INFO: `/v3/projects/5c95be391c4d43a09be9462a4b6a5009/users/11710507ed6f4de1af58980df60dc0eb/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.618898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe0436-7112-4f96-a26e-a1821baac13b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c95be391c4d43a09be9462a4b6a5009, user_id=11710507ed6f4de1af58980df60dc0eb, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.621578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe0436-7112-4f96-a26e-a1821baac13b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.635843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1047/2089] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:42 2022] PUT /identity/v3/projects/5c95be391c4d43a09be9462a4b6a5009/users/11710507ed6f4de1af58980df60dc0eb/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:42.647196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517409c0-c8db-4353-b863-3583f4251513 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.647911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517409c0-c8db-4353-b863-3583f4251513 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.648183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517409c0-c8db-4353-b863-3583f4251513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.683406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-517409c0-c8db-4353-b863-3583f4251513 None None] MFA Rules not processed for user `11710507ed6f4de1af58980df60dc0eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:42.684735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-517409c0-c8db-4353-b863-3583f4251513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:42.690331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe0436-7112-4f96-a26e-a1821baac13b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.692795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-59fe0436-7112-4f96-a26e-a1821baac13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:42.726292 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:42.748205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:42.765450 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1043/2090] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:28:42.767102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe0436-7112-4f96-a26e-a1821baac13b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.769831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517409c0-c8db-4353-b863-3583f4251513 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.772579 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.773615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.773895 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.774170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] PATH_INFO: `/v3/projects/71b097645e824220be4d02f95c561827` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.775401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b097645e824220be4d02f95c561827)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.777385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.780511 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517409c0-c8db-4353-b863-3583f4251513 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.785082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01552e3d-34e8-4aca-8213-3dddfaf70502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.785723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552e3d-34e8-4aca-8213-3dddfaf70502 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.786005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552e3d-34e8-4aca-8213-3dddfaf70502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.786279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552e3d-34e8-4aca-8213-3dddfaf70502 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.786999 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01552e3d-34e8-4aca-8213-3dddfaf70502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.789204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01552e3d-34e8-4aca-8213-3dddfaf70502 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.815368 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1044/2091] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:42 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:42.819927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01552e3d-34e8-4aca-8213-3dddfaf70502 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.825860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b097645e824220be4d02f95c561827', 'request_id': 'req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.831609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b097645e824220be4d02f95c561827', 'request_id': 'req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.839677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b097645e824220be4d02f95c561827', 'request_id': 'req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.853650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] The token cache is being invalidate because project 71b097645e824220be4d02f95c561827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:42.853941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.854642 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-98e7ef55-b48c-4409-b1da-56fd6d6ffb78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:42.858618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1048/2092] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:42 2022] DELETE /identity/v3/projects/71b097645e824220be4d02f95c561827 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:42.861370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d40d99-53ee-40dd-9108-fc4cd9c6340b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.861683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d40d99-53ee-40dd-9108-fc4cd9c6340b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.861958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d40d99-53ee-40dd-9108-fc4cd9c6340b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.883259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d0d40d99-53ee-40dd-9108-fc4cd9c6340b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:42.887480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d0d40d99-53ee-40dd-9108-fc4cd9c6340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:42.888505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01552e3d-34e8-4aca-8213-3dddfaf70502 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.893287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4fb73-d499-4855-b0d1-524d50a643f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.894015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4fb73-d499-4855-b0d1-524d50a643f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.894815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4fb73-d499-4855-b0d1-524d50a643f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.895096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4fb73-d499-4855-b0d1-524d50a643f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.896112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4fb73-d499-4855-b0d1-524d50a643f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.897368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1049/2093] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:28:42.898048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4fb73-d499-4855-b0d1-524d50a643f3 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.904987 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a637a2e-b267-43c0-9a17-39a474a03d45 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.905273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a637a2e-b267-43c0-9a17-39a474a03d45 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.905552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a637a2e-b267-43c0-9a17-39a474a03d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.932153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1045/2094] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:42 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:42.932339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3a637a2e-b267-43c0-9a17-39a474a03d45 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:42.932339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3a637a2e-b267-43c0-9a17-39a474a03d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:42.933220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4fb73-d499-4855-b0d1-524d50a643f3 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.934906 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d1a4fb73-d499-4855-b0d1-524d50a643f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:42.940515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1050/2095] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:28:42.945448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a637a2e-b267-43c0-9a17-39a474a03d45 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.950989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4fb73-d499-4855-b0d1-524d50a643f3 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.961758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a637a2e-b267-43c0-9a17-39a474a03d45 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.963170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8676b4-1594-479b-a519-302c94ff342d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.963846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8676b4-1594-479b-a519-302c94ff342d admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.964126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8676b4-1594-479b-a519-302c94ff342d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.964399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8676b4-1594-479b-a519-302c94ff342d admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.965261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8676b4-1594-479b-a519-302c94ff342d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.966740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8676b4-1594-479b-a519-302c94ff342d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.973913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28a23f-c7ac-4254-a455-5f214e76599c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:42.974606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a23f-c7ac-4254-a455-5f214e76599c admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:42.974893 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a23f-c7ac-4254-a455-5f214e76599c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:42.975167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28a23f-c7ac-4254-a455-5f214e76599c admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:42.975951 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28a23f-c7ac-4254-a455-5f214e76599c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:28:42.976259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28a23f-c7ac-4254-a455-5f214e76599c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:42.977863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28a23f-c7ac-4254-a455-5f214e76599c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:42.979782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1046/2096] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:42.984520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8676b4-1594-479b-a519-302c94ff342d admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:42.994010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1051/2097] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:28:42.996356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8676b4-1594-479b-a519-302c94ff342d admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:42.998187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28a23f-c7ac-4254-a455-5f214e76599c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.001791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df6d41-748b-459a-832d-45bb93d8515d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.002726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6d41-748b-459a-832d-45bb93d8515d admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.003012 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6d41-748b-459a-832d-45bb93d8515d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.003914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df6d41-748b-459a-832d-45bb93d8515d admin admin] PATH_INFO: `/v3/projects/5449a97d3b3e4d65879cdd29e5132438/users/70109a776b0c4f36b179eff8e91a9d7e/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.009141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28a23f-c7ac-4254-a455-5f214e76599c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.016003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed62257-fd85-4c42-a1ab-aae551589850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.016691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed62257-fd85-4c42-a1ab-aae551589850 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.016950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed62257-fd85-4c42-a1ab-aae551589850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.017226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed62257-fd85-4c42-a1ab-aae551589850 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.018147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed62257-fd85-4c42-a1ab-aae551589850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.020923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed62257-fd85-4c42-a1ab-aae551589850 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.022341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df6d41-748b-459a-832d-45bb93d8515d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5449a97d3b3e4d65879cdd29e5132438, user_id=70109a776b0c4f36b179eff8e91a9d7e, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.025156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df6d41-748b-459a-832d-45bb93d8515d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.040260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1047/2098] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:42 2022] PUT /identity/v3/projects/5449a97d3b3e4d65879cdd29e5132438/users/70109a776b0c4f36b179eff8e91a9d7e/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:28:43.044128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df6d41-748b-459a-832d-45bb93d8515d admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.054435 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1052/2099] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:42 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:43.058640 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed62257-fd85-4c42-a1ab-aae551589850 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.118696 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df6d41-748b-459a-832d-45bb93d8515d admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.126773 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:43.128020 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed62257-fd85-4c42-a1ab-aae551589850 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.133756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c0b42-ae18-4eda-8a34-344afaf596d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.134414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c0b42-ae18-4eda-8a34-344afaf596d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.134809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c0b42-ae18-4eda-8a34-344afaf596d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.135089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c0b42-ae18-4eda-8a34-344afaf596d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.135772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c0b42-ae18-4eda-8a34-344afaf596d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.137719 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c0b42-ae18-4eda-8a34-344afaf596d9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.162664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1053/2100] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:43 2022] POST /identity/v3/users => generated 301 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:43.164118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c0b42-ae18-4eda-8a34-344afaf596d9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.174368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c0b42-ae18-4eda-8a34-344afaf596d9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.179180 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35b53e-7b93-474a-90ac-d8a489575c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.179841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35b53e-7b93-474a-90ac-d8a489575c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.180142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35b53e-7b93-474a-90ac-d8a489575c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.180398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35b53e-7b93-474a-90ac-d8a489575c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.181122 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35b53e-7b93-474a-90ac-d8a489575c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.182205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09d9dd-5b80-4b51-a3d9-71493c87fc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.182758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35b53e-7b93-474a-90ac-d8a489575c02 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.182890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09d9dd-5b80-4b51-a3d9-71493c87fc20 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.183162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09d9dd-5b80-4b51-a3d9-71493c87fc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.183435 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09d9dd-5b80-4b51-a3d9-71493c87fc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.184173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09d9dd-5b80-4b51-a3d9-71493c87fc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.185746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09d9dd-5b80-4b51-a3d9-71493c87fc20 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.193077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1054/2101] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:43.196514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1048/2102] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:43.197391 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35b53e-7b93-474a-90ac-d8a489575c02 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.200307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09d9dd-5b80-4b51-a3d9-71493c87fc20 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.207781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35b53e-7b93-474a-90ac-d8a489575c02 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.211329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09d9dd-5b80-4b51-a3d9-71493c87fc20 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.212461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c00225-5ee2-4b3c-84b9-c215641909ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.213108 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c00225-5ee2-4b3c-84b9-c215641909ac admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.213450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c00225-5ee2-4b3c-84b9-c215641909ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.213876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c00225-5ee2-4b3c-84b9-c215641909ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.215084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c00225-5ee2-4b3c-84b9-c215641909ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.216264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37607e53-f0f1-49a1-b196-1b30fdc62ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.216817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c00225-5ee2-4b3c-84b9-c215641909ac admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.217121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37607e53-f0f1-49a1-b196-1b30fdc62ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.217402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37607e53-f0f1-49a1-b196-1b30fdc62ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.217674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37607e53-f0f1-49a1-b196-1b30fdc62ec7 admin admin] PATH_INFO: `/v3/projects/5449a97d3b3e4d65879cdd29e5132438/users/70109a776b0c4f36b179eff8e91a9d7e/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.218196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:43.219959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37607e53-f0f1-49a1-b196-1b30fdc62ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5449a97d3b3e4d65879cdd29e5132438, user_id=70109a776b0c4f36b179eff8e91a9d7e, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.223178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37607e53-f0f1-49a1-b196-1b30fdc62ec7 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.229562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1055/2103] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:43.233586 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c00225-5ee2-4b3c-84b9-c215641909ac admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.234756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1049/2104] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:43 2022] PUT /identity/v3/projects/5449a97d3b3e4d65879cdd29e5132438/users/70109a776b0c4f36b179eff8e91a9d7e/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:28:43.244643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c00225-5ee2-4b3c-84b9-c215641909ac admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.245333 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e1a62-ed98-4e38-a4d6-3016cce700f6 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.245649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e1a62-ed98-4e38-a4d6-3016cce700f6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.245931 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e1a62-ed98-4e38-a4d6-3016cce700f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.276878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-495e1a62-ed98-4e38-a4d6-3016cce700f6 None None] MFA Rules not processed for user `70109a776b0c4f36b179eff8e91a9d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:43.278004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-495e1a62-ed98-4e38-a4d6-3016cce700f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:43.303797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1cbfa5-a83c-43a2-b3c3-68c82bbbf727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.304686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cbfa5-a83c-43a2-b3c3-68c82bbbf727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.304976 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cbfa5-a83c-43a2-b3c3-68c82bbbf727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.305237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1cbfa5-a83c-43a2-b3c3-68c82bbbf727 admin admin] PATH_INFO: `/v3/projects/bacee20cf4364ddda0cceeffaf71669f/users/12626ea5cb904ce89d3aad46667742ee/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.319398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cbfa5-a83c-43a2-b3c3-68c82bbbf727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bacee20cf4364ddda0cceeffaf71669f, user_id=12626ea5cb904ce89d3aad46667742ee, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.322073 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1cbfa5-a83c-43a2-b3c3-68c82bbbf727 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.335685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1056/2105] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:43 2022] PUT /identity/v3/projects/bacee20cf4364ddda0cceeffaf71669f/users/12626ea5cb904ce89d3aad46667742ee/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:28:43.348048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56605f0-5653-4b17-a35c-3a78d56ad0cd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.348365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56605f0-5653-4b17-a35c-3a78d56ad0cd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.348670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56605f0-5653-4b17-a35c-3a78d56ad0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.349664 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1050/2106] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:43.360989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e1a62-ed98-4e38-a4d6-3016cce700f6 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.362376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-495e1a62-ed98-4e38-a4d6-3016cce700f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:43.379261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a56605f0-5653-4b17-a35c-3a78d56ad0cd None None] MFA Rules not processed for user `12626ea5cb904ce89d3aad46667742ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:43.380682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a56605f0-5653-4b17-a35c-3a78d56ad0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:43.427069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e1a62-ed98-4e38-a4d6-3016cce700f6 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.450939 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1057/2107] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:43.455477 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56605f0-5653-4b17-a35c-3a78d56ad0cd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.461145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:43.479844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9d72b-b171-40f8-8c1d-be044e407a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.480805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9d72b-b171-40f8-8c1d-be044e407a6c service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.480805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9d72b-b171-40f8-8c1d-be044e407a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.481026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae9d72b-b171-40f8-8c1d-be044e407a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.482977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9ae9d72b-b171-40f8-8c1d-be044e407a6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:43.519148 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56605f0-5653-4b17-a35c-3a78d56ad0cd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.523969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd9cc9-abf7-45fe-8dd5-026593b8280a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.524628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd9cc9-abf7-45fe-8dd5-026593b8280a admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.524914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd9cc9-abf7-45fe-8dd5-026593b8280a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.525193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd9cc9-abf7-45fe-8dd5-026593b8280a admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.527939 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd9cc9-abf7-45fe-8dd5-026593b8280a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.527939 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd9cc9-abf7-45fe-8dd5-026593b8280a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.541567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:43.542685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9d72b-b171-40f8-8c1d-be044e407a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.544134 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae9d72b-b171-40f8-8c1d-be044e407a6c service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.553904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1051/2108] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:28:43.557167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1058/2109] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:43 2022] POST /identity/v3/projects => generated 401 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:43.561023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9d72b-b171-40f8-8c1d-be044e407a6c service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.573124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae9d72b-b171-40f8-8c1d-be044e407a6c service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.632044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b637a24-3aea-4326-9af9-10d44afd3697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.632721 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b637a24-3aea-4326-9af9-10d44afd3697 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.633005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b637a24-3aea-4326-9af9-10d44afd3697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.633279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b637a24-3aea-4326-9af9-10d44afd3697 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.633998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b637a24-3aea-4326-9af9-10d44afd3697 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.636212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b637a24-3aea-4326-9af9-10d44afd3697 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.658098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1052/2110] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:43 2022] POST /identity/v3/users => generated 307 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:43.662939 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd9cc9-abf7-45fe-8dd5-026593b8280a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.674377 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd9cc9-abf7-45fe-8dd5-026593b8280a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.681300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56bc21-e9fc-446a-836f-7123b2bdb8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.681977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56bc21-e9fc-446a-836f-7123b2bdb8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.682254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56bc21-e9fc-446a-836f-7123b2bdb8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.682581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56bc21-e9fc-446a-836f-7123b2bdb8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.683338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56bc21-e9fc-446a-836f-7123b2bdb8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.685097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56bc21-e9fc-446a-836f-7123b2bdb8f8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.696286 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1059/2111] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:43.702637 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b637a24-3aea-4326-9af9-10d44afd3697 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.719184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b637a24-3aea-4326-9af9-10d44afd3697 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.725583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244201c5-30dc-46c4-b474-b0d73910f13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.726520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244201c5-30dc-46c4-b474-b0d73910f13b admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.726828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244201c5-30dc-46c4-b474-b0d73910f13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.727112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244201c5-30dc-46c4-b474-b0d73910f13b admin admin] PATH_INFO: `/v3/projects/5646e3340f474b189dfa978fe31f3a3b/users/61217a015b8842a3b570c23eb7ba59f9/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.747647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244201c5-30dc-46c4-b474-b0d73910f13b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5646e3340f474b189dfa978fe31f3a3b, user_id=61217a015b8842a3b570c23eb7ba59f9, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.750731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244201c5-30dc-46c4-b474-b0d73910f13b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.767185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1053/2112] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:43 2022] PUT /identity/v3/projects/5646e3340f474b189dfa978fe31f3a3b/users/61217a015b8842a3b570c23eb7ba59f9/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:43.771532 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56bc21-e9fc-446a-836f-7123b2bdb8f8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.782439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56bc21-e9fc-446a-836f-7123b2bdb8f8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.856434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2863b19-9e76-4f76-a6dd-755b19ba54cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.857286 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2863b19-9e76-4f76-a6dd-755b19ba54cb admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.857574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2863b19-9e76-4f76-a6dd-755b19ba54cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.857847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2863b19-9e76-4f76-a6dd-755b19ba54cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.858605 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2863b19-9e76-4f76-a6dd-755b19ba54cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.860265 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2863b19-9e76-4f76-a6dd-755b19ba54cb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.874966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1060/2113] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:43.877100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244201c5-30dc-46c4-b474-b0d73910f13b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:43.887852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244201c5-30dc-46c4-b474-b0d73910f13b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:43.892521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb995f4-d8b4-4fd4-a2be-78e637709bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:43.893528 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb995f4-d8b4-4fd4-a2be-78e637709bf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.893813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb995f4-d8b4-4fd4-a2be-78e637709bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.894577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb995f4-d8b4-4fd4-a2be-78e637709bf5 admin admin] PATH_INFO: `/v3/projects/5646e3340f474b189dfa978fe31f3a3b/users/61217a015b8842a3b570c23eb7ba59f9/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.899497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb995f4-d8b4-4fd4-a2be-78e637709bf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5646e3340f474b189dfa978fe31f3a3b, user_id=61217a015b8842a3b570c23eb7ba59f9, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:43.902177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb995f4-d8b4-4fd4-a2be-78e637709bf5 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:43.915899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1054/2114] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:43 2022] PUT /identity/v3/projects/5646e3340f474b189dfa978fe31f3a3b/users/61217a015b8842a3b570c23eb7ba59f9/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:43.926631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8576a4b-2543-4186-bd7e-0ccb894e468e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:43.926924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8576a4b-2543-4186-bd7e-0ccb894e468e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:43.927229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8576a4b-2543-4186-bd7e-0ccb894e468e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:43.954146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e8576a4b-2543-4186-bd7e-0ccb894e468e None None] MFA Rules not processed for user `61217a015b8842a3b570c23eb7ba59f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:43.955394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e8576a4b-2543-4186-bd7e-0ccb894e468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:44.025247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1061/2115] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:43 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:44.036386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb995f4-d8b4-4fd4-a2be-78e637709bf5 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:44.080351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:44.092344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb995f4-d8b4-4fd4-a2be-78e637709bf5 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:44.095990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e7bfe-e6ca-4f5e-9749-83a283bc9d20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:44.096627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e7bfe-e6ca-4f5e-9749-83a283bc9d20 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:44.096903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e7bfe-e6ca-4f5e-9749-83a283bc9d20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:44.097812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e7bfe-e6ca-4f5e-9749-83a283bc9d20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:44.106772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e7bfe-e6ca-4f5e-9749-83a283bc9d20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:44.108251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e7bfe-e6ca-4f5e-9749-83a283bc9d20 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:44.118196 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1055/2116] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:44 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:44.192724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8576a4b-2543-4186-bd7e-0ccb894e468e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:44.202171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8576a4b-2543-4186-bd7e-0ccb894e468e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:44.205873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf103610-76af-4804-88b8-534aa81f7f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:44.207117 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf103610-76af-4804-88b8-534aa81f7f08 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:44.207484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf103610-76af-4804-88b8-534aa81f7f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:44.207764 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf103610-76af-4804-88b8-534aa81f7f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:44.267497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf103610-76af-4804-88b8-534aa81f7f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:44.268903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf103610-76af-4804-88b8-534aa81f7f08 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:44.279799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1062/2117] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:45.084015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e7bfe-e6ca-4f5e-9749-83a283bc9d20 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:45.092644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e7bfe-e6ca-4f5e-9749-83a283bc9d20 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:45.096345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8e459-a0d7-48c4-ac96-e143a9df0c0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:45.097012 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e459-a0d7-48c4-ac96-e143a9df0c0e service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:45.097298 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e459-a0d7-48c4-ac96-e143a9df0c0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:45.097574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8e459-a0d7-48c4-ac96-e143a9df0c0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:45.099396 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a6a8e459-a0d7-48c4-ac96-e143a9df0c0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:45.162269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8e459-a0d7-48c4-ac96-e143a9df0c0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:45.163708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8e459-a0d7-48c4-ac96-e143a9df0c0e service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:45.172518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1056/2118] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:45.842753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf103610-76af-4804-88b8-534aa81f7f08 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:45.845265 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cf103610-76af-4804-88b8-534aa81f7f08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:45.856378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:45.862513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf103610-76af-4804-88b8-534aa81f7f08 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:45.869428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3070bf-8bb7-4a0f-bdcb-8beb722f1895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:45.870462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3070bf-8bb7-4a0f-bdcb-8beb722f1895 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:45.870833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3070bf-8bb7-4a0f-bdcb-8beb722f1895 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:45.871195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3070bf-8bb7-4a0f-bdcb-8beb722f1895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:45.938834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3070bf-8bb7-4a0f-bdcb-8beb722f1895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:45.940289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3070bf-8bb7-4a0f-bdcb-8beb722f1895 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:45.951359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1063/2119] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:46.492499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8e459-a0d7-48c4-ac96-e143a9df0c0e service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:46.493977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a6a8e459-a0d7-48c4-ac96-e143a9df0c0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:46.551566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8e459-a0d7-48c4-ac96-e143a9df0c0e service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:46.555059 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a466e-4f23-4326-990f-b54c6164a774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:46.555703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a466e-4f23-4326-990f-b54c6164a774 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:46.555982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a466e-4f23-4326-990f-b54c6164a774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:46.556254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a466e-4f23-4326-990f-b54c6164a774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:46.558100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1e4a466e-4f23-4326-990f-b54c6164a774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:46.625094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a466e-4f23-4326-990f-b54c6164a774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:46.626566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a466e-4f23-4326-990f-b54c6164a774 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:46.636824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1057/2120] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:48.971303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3070bf-8bb7-4a0f-bdcb-8beb722f1895 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:49.095788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3070bf-8bb7-4a0f-bdcb-8beb722f1895 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:49.108682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:49.109577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:49.109864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:49.110142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] PATH_INFO: `/v3/users/12626ea5cb904ce89d3aad46667742ee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:49.116566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12626ea5cb904ce89d3aad46667742ee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:49.119073 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:49.224265 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12626ea5cb904ce89d3aad46667742ee'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:49.253783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12626ea5cb904ce89d3aad46667742ee'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:49.254752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12626ea5cb904ce89d3aad46667742ee'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:49.265441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12626ea5cb904ce89d3aad46667742ee'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:49.275677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12626ea5cb904ce89d3aad46667742ee'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:49.294381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1064/2121] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:48 2022] DELETE /identity/v3/users/12626ea5cb904ce89d3aad46667742ee => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:49.299071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb817d5b-fce3-4c08-bfcd-a2c09503286e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:49.299071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb817d5b-fce3-4c08-bfcd-a2c09503286e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:49.299487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb817d5b-fce3-4c08-bfcd-a2c09503286e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:49.332752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-cb817d5b-fce3-4c08-bfcd-a2c09503286e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:49.334024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-cb817d5b-fce3-4c08-bfcd-a2c09503286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:49.415437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:49.541543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1058/2122] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:49.550779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:49.552470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:49.593829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:49.717054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290e6ee-aaaf-44b5-b168-42b16f0825c4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:49.720621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a6992-86c5-4aeb-afe2-3b06f0bf4185 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:49.722423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a6992-86c5-4aeb-afe2-3b06f0bf4185 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:49.722423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a6992-86c5-4aeb-afe2-3b06f0bf4185 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:49.723258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a6992-86c5-4aeb-afe2-3b06f0bf4185 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:49.735149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a6992-86c5-4aeb-afe2-3b06f0bf4185 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:49.738414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a6992-86c5-4aeb-afe2-3b06f0bf4185 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:49.750778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1065/2123] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:28:50.133977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb817d5b-fce3-4c08-bfcd-a2c09503286e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:50.135866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-cb817d5b-fce3-4c08-bfcd-a2c09503286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:50.157343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb817d5b-fce3-4c08-bfcd-a2c09503286e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:50.174549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:50.175475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:50.175774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:50.176042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] PATH_INFO: `/v3/projects/bacee20cf4364ddda0cceeffaf71669f` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:50.177421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bacee20cf4364ddda0cceeffaf71669f)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:50.179826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:50.260216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bacee20cf4364ddda0cceeffaf71669f', 'request_id': 'req-8b2e454f-7764-4999-bb2e-24e1078353b3'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.267792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bacee20cf4364ddda0cceeffaf71669f', 'request_id': 'req-8b2e454f-7764-4999-bb2e-24e1078353b3'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.278326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bacee20cf4364ddda0cceeffaf71669f', 'request_id': 'req-8b2e454f-7764-4999-bb2e-24e1078353b3'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.295449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] The token cache is being invalidate because project bacee20cf4364ddda0cceeffaf71669f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:50.296097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.296986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.299817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1059/2124] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:50 2022] DELETE /identity/v3/projects/bacee20cf4364ddda0cceeffaf71669f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:50.304738 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a6992-86c5-4aeb-afe2-3b06f0bf4185 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:50.315229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a6992-86c5-4aeb-afe2-3b06f0bf4185 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:50.437760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7f435-dd5f-4b7b-abe3-1462285c7012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:50.440058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7f435-dd5f-4b7b-abe3-1462285c7012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:50.441223 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7f435-dd5f-4b7b-abe3-1462285c7012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:50.441638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7f435-dd5f-4b7b-abe3-1462285c7012 admin admin] PATH_INFO: `/v3/users/61217a015b8842a3b570c23eb7ba59f9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:50.444245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7f435-dd5f-4b7b-abe3-1462285c7012 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61217a015b8842a3b570c23eb7ba59f9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:50.448727 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7f435-dd5f-4b7b-abe3-1462285c7012 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:50.584007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:50.585719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:50.596872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-63a7f435-dd5f-4b7b-abe3-1462285c7012 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61217a015b8842a3b570c23eb7ba59f9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.624915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-63a7f435-dd5f-4b7b-abe3-1462285c7012 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61217a015b8842a3b570c23eb7ba59f9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.625414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-63a7f435-dd5f-4b7b-abe3-1462285c7012 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61217a015b8842a3b570c23eb7ba59f9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.632852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e454f-7764-4999-bb2e-24e1078353b3 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:50.654740 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:50.659387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:50.667190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:50.667478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] PATH_INFO: `/v3/users/1f607d90947a49a3ae38c4e8de636474` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:50.668716 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-63a7f435-dd5f-4b7b-abe3-1462285c7012 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61217a015b8842a3b570c23eb7ba59f9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.669449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-63a7f435-dd5f-4b7b-abe3-1462285c7012 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61217a015b8842a3b570c23eb7ba59f9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.675800 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f607d90947a49a3ae38c4e8de636474)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:50.678550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:50.697699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1066/2125] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:50 2022] DELETE /identity/v3/users/61217a015b8842a3b570c23eb7ba59f9 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:50.771666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f607d90947a49a3ae38c4e8de636474'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.785627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f607d90947a49a3ae38c4e8de636474'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.786682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f607d90947a49a3ae38c4e8de636474'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.795127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f607d90947a49a3ae38c4e8de636474'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.796639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f607d90947a49a3ae38c4e8de636474'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:50.817073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1060/2126] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:50 2022] DELETE /identity/v3/users/1f607d90947a49a3ae38c4e8de636474 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:50.822011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550892f-ed40-4448-91d4-c01cdde3c961 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:50.822306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550892f-ed40-4448-91d4-c01cdde3c961 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:50.822602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550892f-ed40-4448-91d4-c01cdde3c961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:50.844958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7550892f-ed40-4448-91d4-c01cdde3c961 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:50.846343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7550892f-ed40-4448-91d4-c01cdde3c961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:50.893082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:50.932790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1067/2127] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:50.964611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:50.966587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:51.027444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:51.059764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2c0db-00cb-4bc4-95e7-8d450cbfb4d4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:51.065125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670aca0-4bce-40ef-bdc4-be73846cc4b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:51.066457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670aca0-4bce-40ef-bdc4-be73846cc4b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:51.066772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670aca0-4bce-40ef-bdc4-be73846cc4b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:51.067048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670aca0-4bce-40ef-bdc4-be73846cc4b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:51.077840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670aca0-4bce-40ef-bdc4-be73846cc4b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:51.079369 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670aca0-4bce-40ef-bdc4-be73846cc4b7 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:51.090374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1061/2128] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:51.600096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550892f-ed40-4448-91d4-c01cdde3c961 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:51.602424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7550892f-ed40-4448-91d4-c01cdde3c961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:51.630377 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550892f-ed40-4448-91d4-c01cdde3c961 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:51.636578 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:51.637793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:51.638324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:51.638841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] PATH_INFO: `/v3/projects/5646e3340f474b189dfa978fe31f3a3b` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:51.640360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5646e3340f474b189dfa978fe31f3a3b)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:51.643015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:51.726681 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5646e3340f474b189dfa978fe31f3a3b', 'request_id': 'req-59db6301-13c6-420b-978d-f973aef7d0b4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:51.732345 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5646e3340f474b189dfa978fe31f3a3b', 'request_id': 'req-59db6301-13c6-420b-978d-f973aef7d0b4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:51.741579 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5646e3340f474b189dfa978fe31f3a3b', 'request_id': 'req-59db6301-13c6-420b-978d-f973aef7d0b4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:51.754189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] The token cache is being invalidate because project 5646e3340f474b189dfa978fe31f3a3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:51.754478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:51.755148 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-59db6301-13c6-420b-978d-f973aef7d0b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:51.756696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1068/2129] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:51 2022] DELETE /identity/v3/projects/5646e3340f474b189dfa978fe31f3a3b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:51.761713 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c87c9-3691-42a9-a5b9-75d6fabeff72 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:51.762222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c87c9-3691-42a9-a5b9-75d6fabeff72 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:51.762462 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c87c9-3691-42a9-a5b9-75d6fabeff72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:51.795266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-326c87c9-3691-42a9-a5b9-75d6fabeff72 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:51.796815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-326c87c9-3691-42a9-a5b9-75d6fabeff72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:51.884771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1062/2130] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:51.893021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1671a6-c9c2-4f67-80e1-3a19e977e88e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:51.893343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1671a6-c9c2-4f67-80e1-3a19e977e88e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:51.895568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1671a6-c9c2-4f67-80e1-3a19e977e88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:51.922023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6a1671a6-c9c2-4f67-80e1-3a19e977e88e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:51.923556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6a1671a6-c9c2-4f67-80e1-3a19e977e88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:51.938384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1069/2131] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:51.939293 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c87c9-3691-42a9-a5b9-75d6fabeff72 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:51.964476 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c87c9-3691-42a9-a5b9-75d6fabeff72 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:51.967889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1671a6-c9c2-4f67-80e1-3a19e977e88e None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:51.971404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6a1671a6-c9c2-4f67-80e1-3a19e977e88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:51.980176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547c399-496b-49e1-a37f-15285e61144b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:51.981255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547c399-496b-49e1-a37f-15285e61144b admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:51.981773 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547c399-496b-49e1-a37f-15285e61144b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:51.982502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547c399-496b-49e1-a37f-15285e61144b admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:51.983509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547c399-496b-49e1-a37f-15285e61144b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:28:51.985046 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547c399-496b-49e1-a37f-15285e61144b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:51.989011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1671a6-c9c2-4f67-80e1-3a19e977e88e None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:51.990774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547c399-496b-49e1-a37f-15285e61144b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:51.996430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:51.997561 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:51.998031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:51.999088 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] PATH_INFO: `/v3/projects/eed335082a544fb88ece841ae1035aac` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:52.001250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed335082a544fb88ece841ae1035aac)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:52.005218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:52.013248 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1063/2132] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:28:52.014239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547c399-496b-49e1-a37f-15285e61144b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:52.039808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547c399-496b-49e1-a37f-15285e61144b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:52.059626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d76e76-85c5-4242-8c3c-6b95d964971c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:52.060443 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d76e76-85c5-4242-8c3c-6b95d964971c admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:52.060960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d76e76-85c5-4242-8c3c-6b95d964971c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:52.061833 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d76e76-85c5-4242-8c3c-6b95d964971c admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:52.062791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d76e76-85c5-4242-8c3c-6b95d964971c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:52.065675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d76e76-85c5-4242-8c3c-6b95d964971c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:52.103101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed335082a544fb88ece841ae1035aac', 'request_id': 'req-49c95d31-210b-4722-bfdf-470277dc88bc'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:52.109909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed335082a544fb88ece841ae1035aac', 'request_id': 'req-49c95d31-210b-4722-bfdf-470277dc88bc'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:52.120683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1064/2133] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:52 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:52.122402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d76e76-85c5-4242-8c3c-6b95d964971c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:52.126187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed335082a544fb88ece841ae1035aac', 'request_id': 'req-49c95d31-210b-4722-bfdf-470277dc88bc'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:52.146963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] The token cache is being invalidate because project eed335082a544fb88ece841ae1035aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:52.147173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:52.148330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:52.154448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1070/2134] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:51 2022] DELETE /identity/v3/projects/eed335082a544fb88ece841ae1035aac => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:52.155782 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:52.157317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:52.224943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:52.234392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d76e76-85c5-4242-8c3c-6b95d964971c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:52.242853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54602c23-e036-4dff-a0c4-baef138306aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:52.243940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54602c23-e036-4dff-a0c4-baef138306aa admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:52.243940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54602c23-e036-4dff-a0c4-baef138306aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:52.244164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54602c23-e036-4dff-a0c4-baef138306aa admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:52.244803 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54602c23-e036-4dff-a0c4-baef138306aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:52.247036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54602c23-e036-4dff-a0c4-baef138306aa admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:52.270559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c95d31-210b-4722-bfdf-470277dc88bc admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:52.278178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6897efe-7d86-4794-a696-5daf5e66b528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:52.279273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6897efe-7d86-4794-a696-5daf5e66b528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:52.279558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6897efe-7d86-4794-a696-5daf5e66b528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:52.279843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6897efe-7d86-4794-a696-5daf5e66b528 admin admin] PATH_INFO: `/v3/users/11710507ed6f4de1af58980df60dc0eb` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:52.282078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6897efe-7d86-4794-a696-5daf5e66b528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11710507ed6f4de1af58980df60dc0eb)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:52.284873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6897efe-7d86-4794-a696-5daf5e66b528 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:52.296305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1065/2135] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:52 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:52.297611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54602c23-e036-4dff-a0c4-baef138306aa admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:52.302752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-54602c23-e036-4dff-a0c4-baef138306aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:52.325357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54602c23-e036-4dff-a0c4-baef138306aa admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:52.332397 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c697e82-94f7-482d-b283-b0d2739c6d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:52.334101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c697e82-94f7-482d-b283-b0d2739c6d2e admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:52.334101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c697e82-94f7-482d-b283-b0d2739c6d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:52.334101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c697e82-94f7-482d-b283-b0d2739c6d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:52.334809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c697e82-94f7-482d-b283-b0d2739c6d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:52.336527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c697e82-94f7-482d-b283-b0d2739c6d2e admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:52.363448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1066/2136] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:52.368725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c697e82-94f7-482d-b283-b0d2739c6d2e admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:52.381973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c697e82-94f7-482d-b283-b0d2739c6d2e admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:52.390224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e5a29-be19-449f-9385-0179705bffba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:52.391345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e5a29-be19-449f-9385-0179705bffba admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:52.391782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e5a29-be19-449f-9385-0179705bffba admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:52.392182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e5a29-be19-449f-9385-0179705bffba admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:52.392951 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e5a29-be19-449f-9385-0179705bffba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:52.406999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a6897efe-7d86-4794-a696-5daf5e66b528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11710507ed6f4de1af58980df60dc0eb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:52.407720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e5a29-be19-449f-9385-0179705bffba admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:52.412208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:52.443043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a6897efe-7d86-4794-a696-5daf5e66b528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11710507ed6f4de1af58980df60dc0eb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:52.445035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a6897efe-7d86-4794-a696-5daf5e66b528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11710507ed6f4de1af58980df60dc0eb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:52.449623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1067/2137] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:52.452531 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e5a29-be19-449f-9385-0179705bffba admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:52.458519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a6897efe-7d86-4794-a696-5daf5e66b528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11710507ed6f4de1af58980df60dc0eb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:52.461812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a6897efe-7d86-4794-a696-5daf5e66b528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11710507ed6f4de1af58980df60dc0eb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:52.471061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e5a29-be19-449f-9385-0179705bffba admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:52.477474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c14137-3a34-41d2-9da2-6e0ccf53177f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:52.478463 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14137-3a34-41d2-9da2-6e0ccf53177f admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:52.478770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14137-3a34-41d2-9da2-6e0ccf53177f admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:52.479048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c14137-3a34-41d2-9da2-6e0ccf53177f admin admin] PATH_INFO: `/v3/projects/ab83cd247aae48658343b52330a0f0be/users/24eaf794f44c4e9d8969bb77950ab0f0/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:52.495811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1071/2138] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:52 2022] DELETE /identity/v3/users/11710507ed6f4de1af58980df60dc0eb => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:52.503671 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c14137-3a34-41d2-9da2-6e0ccf53177f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab83cd247aae48658343b52330a0f0be, user_id=24eaf794f44c4e9d8969bb77950ab0f0, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:52.507339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c14137-3a34-41d2-9da2-6e0ccf53177f admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:52.537014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1068/2139] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:52 2022] PUT /identity/v3/projects/ab83cd247aae48658343b52330a0f0be/users/24eaf794f44c4e9d8969bb77950ab0f0/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:52.538856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba8cf-7bbb-4057-8ca1-8b86e6f2b05d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:52.539146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba8cf-7bbb-4057-8ca1-8b86e6f2b05d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:52.540716 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ba8cf-7bbb-4057-8ca1-8b86e6f2b05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:52.585747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b67ba8cf-7bbb-4057-8ca1-8b86e6f2b05d None None] MFA Rules not processed for user `24eaf794f44c4e9d8969bb77950ab0f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:52.588030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b67ba8cf-7bbb-4057-8ca1-8b86e6f2b05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:52.814909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1072/2140] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:52.821533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c14137-3a34-41d2-9da2-6e0ccf53177f admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:52.824200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d5c14137-3a34-41d2-9da2-6e0ccf53177f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:52.926548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c14137-3a34-41d2-9da2-6e0ccf53177f admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:52.948615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004376bb-6347-4685-b7d4-4cb433bc0fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:52.957475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004376bb-6347-4685-b7d4-4cb433bc0fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:52.958014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004376bb-6347-4685-b7d4-4cb433bc0fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:52.958451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004376bb-6347-4685-b7d4-4cb433bc0fe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:52.959386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004376bb-6347-4685-b7d4-4cb433bc0fe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:52.962262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004376bb-6347-4685-b7d4-4cb433bc0fe0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:53.025373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1069/2141] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:52 2022] POST /identity/v3/projects => generated 387 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:53.027115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ba8cf-7bbb-4057-8ca1-8b86e6f2b05d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:53.046184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ba8cf-7bbb-4057-8ca1-8b86e6f2b05d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:53.102462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:53.121972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fc1c0-8b55-483b-9598-84e757291c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:53.124048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fc1c0-8b55-483b-9598-84e757291c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:53.124697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fc1c0-8b55-483b-9598-84e757291c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:53.124942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1fc1c0-8b55-483b-9598-84e757291c8d admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:53.125639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fc1c0-8b55-483b-9598-84e757291c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:53.129728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1fc1c0-8b55-483b-9598-84e757291c8d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:53.168168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1073/2142] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:53 2022] POST /identity/v3/users => generated 300 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:53.170122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004376bb-6347-4685-b7d4-4cb433bc0fe0 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:53.183537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004376bb-6347-4685-b7d4-4cb433bc0fe0 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:53.194468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b877d-ab74-4f2f-9bc1-7e94486fe6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:53.195533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b877d-ab74-4f2f-9bc1-7e94486fe6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:53.195824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b877d-ab74-4f2f-9bc1-7e94486fe6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:53.196106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b877d-ab74-4f2f-9bc1-7e94486fe6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:53.196848 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b877d-ab74-4f2f-9bc1-7e94486fe6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:53.198902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b877d-ab74-4f2f-9bc1-7e94486fe6f9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:53.216846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1070/2143] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:53.218847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fc1c0-8b55-483b-9598-84e757291c8d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:53.234563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1fc1c0-8b55-483b-9598-84e757291c8d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:53.247413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eff2f3-78cc-49b9-b37d-46ba5aea4b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:53.248460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eff2f3-78cc-49b9-b37d-46ba5aea4b2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:53.248982 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eff2f3-78cc-49b9-b37d-46ba5aea4b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:53.249407 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eff2f3-78cc-49b9-b37d-46ba5aea4b2c admin admin] PATH_INFO: `/v3/projects/ea1b7c268dc84c31b630d73e071700c3/users/bf1f4cb0fe2647839353b7f5958a3596/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:53.267818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eff2f3-78cc-49b9-b37d-46ba5aea4b2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea1b7c268dc84c31b630d73e071700c3, user_id=bf1f4cb0fe2647839353b7f5958a3596, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:53.272098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eff2f3-78cc-49b9-b37d-46ba5aea4b2c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:53.293429 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1074/2144] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:53 2022] PUT /identity/v3/projects/ea1b7c268dc84c31b630d73e071700c3/users/bf1f4cb0fe2647839353b7f5958a3596/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:53.297086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b877d-ab74-4f2f-9bc1-7e94486fe6f9 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:53.314596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b877d-ab74-4f2f-9bc1-7e94486fe6f9 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:53.431514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579309d-66bb-4947-b4fa-9b20c1c742fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:53.432210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579309d-66bb-4947-b4fa-9b20c1c742fa admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:53.432491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579309d-66bb-4947-b4fa-9b20c1c742fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:53.432766 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579309d-66bb-4947-b4fa-9b20c1c742fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:53.446132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579309d-66bb-4947-b4fa-9b20c1c742fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:53.448110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579309d-66bb-4947-b4fa-9b20c1c742fa admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:53.469540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1071/2145] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:53.473989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eff2f3-78cc-49b9-b37d-46ba5aea4b2c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:53.500768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eff2f3-78cc-49b9-b37d-46ba5aea4b2c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:53.512363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44903cd7-60c7-4f96-909c-559f31cf9ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:53.513473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44903cd7-60c7-4f96-909c-559f31cf9ebe admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:53.513854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44903cd7-60c7-4f96-909c-559f31cf9ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:53.514229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44903cd7-60c7-4f96-909c-559f31cf9ebe admin admin] PATH_INFO: `/v3/projects/ea1b7c268dc84c31b630d73e071700c3/users/bf1f4cb0fe2647839353b7f5958a3596/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:53.517236 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44903cd7-60c7-4f96-909c-559f31cf9ebe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea1b7c268dc84c31b630d73e071700c3, user_id=bf1f4cb0fe2647839353b7f5958a3596, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:53.524385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44903cd7-60c7-4f96-909c-559f31cf9ebe admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:53.546371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1075/2146] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:53 2022] PUT /identity/v3/projects/ea1b7c268dc84c31b630d73e071700c3/users/bf1f4cb0fe2647839353b7f5958a3596/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:53.548241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400c0a8-f1a9-41d8-9cd4-0d2ec282edc6 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:53.548655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400c0a8-f1a9-41d8-9cd4-0d2ec282edc6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:53.549076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400c0a8-f1a9-41d8-9cd4-0d2ec282edc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:53.581196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44903cd7-60c7-4f96-909c-559f31cf9ebe admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:53.583362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e400c0a8-f1a9-41d8-9cd4-0d2ec282edc6 None None] MFA Rules not processed for user `bf1f4cb0fe2647839353b7f5958a3596`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:53.584524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-44903cd7-60c7-4f96-909c-559f31cf9ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:53.584900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e400c0a8-f1a9-41d8-9cd4-0d2ec282edc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:53.624523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:53.691383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44903cd7-60c7-4f96-909c-559f31cf9ebe admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:53.699205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:53.700475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:53.701707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:53.702173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] PATH_INFO: `/v3/projects/5c95be391c4d43a09be9462a4b6a5009` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:53.704440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c95be391c4d43a09be9462a4b6a5009)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:53.707409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1072/2147] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:53.708883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:53.718651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400c0a8-f1a9-41d8-9cd4-0d2ec282edc6 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:53.720689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e400c0a8-f1a9-41d8-9cd4-0d2ec282edc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:53.803281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c95be391c4d43a09be9462a4b6a5009', 'request_id': 'req-9098898b-3693-4717-ab28-287a0d8c604f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:53.811009 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c95be391c4d43a09be9462a4b6a5009', 'request_id': 'req-9098898b-3693-4717-ab28-287a0d8c604f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:53.811889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400c0a8-f1a9-41d8-9cd4-0d2ec282edc6 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:53.823212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c95be391c4d43a09be9462a4b6a5009', 'request_id': 'req-9098898b-3693-4717-ab28-287a0d8c604f'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:53.863800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] The token cache is being invalidate because project 5c95be391c4d43a09be9462a4b6a5009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:53.863800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:53.865698 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:53.870785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1076/2148] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:53 2022] DELETE /identity/v3/projects/5c95be391c4d43a09be9462a4b6a5009 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:53.874195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:53.874195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:53.916987 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022167-a4e1-4a64-aedd-991698271e22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:53.917704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022167-a4e1-4a64-aedd-991698271e22 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:53.917970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022167-a4e1-4a64-aedd-991698271e22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:53.918246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a022167-a4e1-4a64-aedd-991698271e22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:53.928985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0a022167-a4e1-4a64-aedd-991698271e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:53.949552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:54.027242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9098898b-3693-4717-ab28-287a0d8c604f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:54.034100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c457f8-84f2-4f31-9931-193c691b61f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:54.035089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c457f8-84f2-4f31-9931-193c691b61f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:54.035375 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c457f8-84f2-4f31-9931-193c691b61f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:54.035653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c457f8-84f2-4f31-9931-193c691b61f5 admin admin] PATH_INFO: `/v3/users/70109a776b0c4f36b179eff8e91a9d7e` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:54.036885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c457f8-84f2-4f31-9931-193c691b61f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70109a776b0c4f36b179eff8e91a9d7e)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:54.038885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c457f8-84f2-4f31-9931-193c691b61f5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:54.059032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a022167-a4e1-4a64-aedd-991698271e22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:54.061611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a022167-a4e1-4a64-aedd-991698271e22 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:54.074422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1073/2149] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:28:54.137317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-83c457f8-84f2-4f31-9931-193c691b61f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70109a776b0c4f36b179eff8e91a9d7e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.171825 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-83c457f8-84f2-4f31-9931-193c691b61f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70109a776b0c4f36b179eff8e91a9d7e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.172805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-83c457f8-84f2-4f31-9931-193c691b61f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70109a776b0c4f36b179eff8e91a9d7e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.183473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-83c457f8-84f2-4f31-9931-193c691b61f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70109a776b0c4f36b179eff8e91a9d7e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.184153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-83c457f8-84f2-4f31-9931-193c691b61f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70109a776b0c4f36b179eff8e91a9d7e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.220971 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1077/2150] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:53 2022] DELETE /identity/v3/users/70109a776b0c4f36b179eff8e91a9d7e => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:54.244608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022167-a4e1-4a64-aedd-991698271e22 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:54.246472 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0a022167-a4e1-4a64-aedd-991698271e22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:54.260857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a022167-a4e1-4a64-aedd-991698271e22 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:54.306668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:54.340746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:54.341645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:54.341930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:54.342580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] PATH_INFO: `/v3/users/24eaf794f44c4e9d8969bb77950ab0f0` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:54.344268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24eaf794f44c4e9d8969bb77950ab0f0)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:54.346251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:54.409687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24eaf794f44c4e9d8969bb77950ab0f0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.425623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24eaf794f44c4e9d8969bb77950ab0f0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.426879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24eaf794f44c4e9d8969bb77950ab0f0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.436693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24eaf794f44c4e9d8969bb77950ab0f0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.438267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24eaf794f44c4e9d8969bb77950ab0f0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.455877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1074/2151] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:54 2022] DELETE /identity/v3/users/24eaf794f44c4e9d8969bb77950ab0f0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:54.461050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f053f1-732e-4f7f-b7d2-e7e6f34d325d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:54.461715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f053f1-732e-4f7f-b7d2-e7e6f34d325d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:54.462279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f053f1-732e-4f7f-b7d2-e7e6f34d325d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:54.496323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-33f053f1-732e-4f7f-b7d2-e7e6f34d325d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:54.498379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-33f053f1-732e-4f7f-b7d2-e7e6f34d325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:54.608307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1078/2152] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:54.623342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:54.625120 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:54.717145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8192ff91-ca6a-4603-8640-46638248a485 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:54.721490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edfa70a-83b0-4b74-80bd-446e13a14a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:54.722517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfa70a-83b0-4b74-80bd-446e13a14a9c service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:54.722517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfa70a-83b0-4b74-80bd-446e13a14a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:54.723056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edfa70a-83b0-4b74-80bd-446e13a14a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:54.736769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfa70a-83b0-4b74-80bd-446e13a14a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:54.738852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edfa70a-83b0-4b74-80bd-446e13a14a9c service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:54.758278 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1075/2153] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:28:54.775107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f053f1-732e-4f7f-b7d2-e7e6f34d325d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:54.776836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-33f053f1-732e-4f7f-b7d2-e7e6f34d325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:54.792975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f053f1-732e-4f7f-b7d2-e7e6f34d325d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:54.798140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:54.799597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:54.800800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:54.801562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] PATH_INFO: `/v3/projects/5449a97d3b3e4d65879cdd29e5132438` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:54.804112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5449a97d3b3e4d65879cdd29e5132438)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:54.807276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:54.870158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:54.871959 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5449a97d3b3e4d65879cdd29e5132438', 'request_id': 'req-daadd08e-7087-4bff-aac2-f070790e9b50'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.879461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5449a97d3b3e4d65879cdd29e5132438', 'request_id': 'req-daadd08e-7087-4bff-aac2-f070790e9b50'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.890126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5449a97d3b3e4d65879cdd29e5132438', 'request_id': 'req-daadd08e-7087-4bff-aac2-f070790e9b50'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.912457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] The token cache is being invalidate because project 5449a97d3b3e4d65879cdd29e5132438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:54.912871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.914293 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-daadd08e-7087-4bff-aac2-f070790e9b50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:54.918638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1079/2154] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:54 2022] DELETE /identity/v3/projects/5449a97d3b3e4d65879cdd29e5132438 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:54.922292 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5b71a-99e2-43d3-af7c-f06725ea3be5 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:54.922943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5b71a-99e2-43d3-af7c-f06725ea3be5 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:54.923273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5b71a-99e2-43d3-af7c-f06725ea3be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:54.945956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-51d5b71a-99e2-43d3-af7c-f06725ea3be5 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:54.947320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-51d5b71a-99e2-43d3-af7c-f06725ea3be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:55.007598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:55.035805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1076/2155] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:28:55.044069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee19584-0ae4-4abc-87c5-f34b26c170c2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:55.044426 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee19584-0ae4-4abc-87c5-f34b26c170c2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:55.044771 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee19584-0ae4-4abc-87c5-f34b26c170c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:55.087116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-aee19584-0ae4-4abc-87c5-f34b26c170c2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:55.091237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aee19584-0ae4-4abc-87c5-f34b26c170c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:55.117470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1080/2156] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:55.120059 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5b71a-99e2-43d3-af7c-f06725ea3be5 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:55.133226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5b71a-99e2-43d3-af7c-f06725ea3be5 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:55.144525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da329c-381a-4e8f-bc7a-4624392ccffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:55.170191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da329c-381a-4e8f-bc7a-4624392ccffd admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:55.170574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da329c-381a-4e8f-bc7a-4624392ccffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:55.170869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da329c-381a-4e8f-bc7a-4624392ccffd admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:55.172954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da329c-381a-4e8f-bc7a-4624392ccffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:28:55.173944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da329c-381a-4e8f-bc7a-4624392ccffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:55.189778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da329c-381a-4e8f-bc7a-4624392ccffd admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:55.201635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1077/2157] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:28:55.205490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee19584-0ae4-4abc-87c5-f34b26c170c2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:55.218670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee19584-0ae4-4abc-87c5-f34b26c170c2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:55.225958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2f29d-42e0-4ab8-912c-aa68a40155c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:55.226674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2f29d-42e0-4ab8-912c-aa68a40155c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:55.226960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2f29d-42e0-4ab8-912c-aa68a40155c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:55.227300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2f29d-42e0-4ab8-912c-aa68a40155c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:55.228007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2f29d-42e0-4ab8-912c-aa68a40155c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:55.231083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2f29d-42e0-4ab8-912c-aa68a40155c4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:55.299255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1081/2158] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:55 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:55.301295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da329c-381a-4e8f-bc7a-4624392ccffd admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:55.318429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da329c-381a-4e8f-bc7a-4624392ccffd admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:55.437978 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ea46c-6a4f-4207-a8a6-f07f3a2b0706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:55.438932 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ea46c-6a4f-4207-a8a6-f07f3a2b0706 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:55.439293 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ea46c-6a4f-4207-a8a6-f07f3a2b0706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:55.440213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ea46c-6a4f-4207-a8a6-f07f3a2b0706 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:55.441025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ea46c-6a4f-4207-a8a6-f07f3a2b0706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:55.443259 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ea46c-6a4f-4207-a8a6-f07f3a2b0706 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:55.468707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1078/2159] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:55 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:55.472822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2f29d-42e0-4ab8-912c-aa68a40155c4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:55.485585 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2f29d-42e0-4ab8-912c-aa68a40155c4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:55.492360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef160b-4eb8-48c2-ae42-2c202b8d4e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:55.493135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef160b-4eb8-48c2-ae42-2c202b8d4e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:55.494841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef160b-4eb8-48c2-ae42-2c202b8d4e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:55.495196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef160b-4eb8-48c2-ae42-2c202b8d4e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:55.496041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef160b-4eb8-48c2-ae42-2c202b8d4e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:55.499808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef160b-4eb8-48c2-ae42-2c202b8d4e18 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:55.516566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1082/2160] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:55.522587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ea46c-6a4f-4207-a8a6-f07f3a2b0706 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:55.538203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ea46c-6a4f-4207-a8a6-f07f3a2b0706 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:55.544109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308b745-f95f-42c9-9ba5-5395efcdc38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:55.545495 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308b745-f95f-42c9-9ba5-5395efcdc38d admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:55.546236 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308b745-f95f-42c9-9ba5-5395efcdc38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:55.546236 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308b745-f95f-42c9-9ba5-5395efcdc38d admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:55.546906 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308b745-f95f-42c9-9ba5-5395efcdc38d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:55.548644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308b745-f95f-42c9-9ba5-5395efcdc38d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:55.570435 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1079/2161] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:55.571929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef160b-4eb8-48c2-ae42-2c202b8d4e18 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:55.589364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef160b-4eb8-48c2-ae42-2c202b8d4e18 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:55.598356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb4c3cd-a37e-46ae-8bcc-5af771b15300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:55.599486 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4c3cd-a37e-46ae-8bcc-5af771b15300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:55.599695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4c3cd-a37e-46ae-8bcc-5af771b15300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:55.599937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4c3cd-a37e-46ae-8bcc-5af771b15300 admin admin] PATH_INFO: `/v3/projects/fca5bfd4ce91447ba23b602b645d1f7f/users/40c7c4699490462b97482053134ac70d/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:55.602850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308b745-f95f-42c9-9ba5-5395efcdc38d admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:55.606095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0308b745-f95f-42c9-9ba5-5395efcdc38d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:55.620093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4c3cd-a37e-46ae-8bcc-5af771b15300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca5bfd4ce91447ba23b602b645d1f7f, user_id=40c7c4699490462b97482053134ac70d, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:55.623857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb4c3cd-a37e-46ae-8bcc-5af771b15300 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:55.625191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308b745-f95f-42c9-9ba5-5395efcdc38d admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:55.633224 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:55.634598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:55.634925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:55.635445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] PATH_INFO: `/v3/projects/ab83cd247aae48658343b52330a0f0be` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:55.636755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab83cd247aae48658343b52330a0f0be)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:55.638912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:55.647587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1083/2162] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:55 2022] PUT /identity/v3/projects/fca5bfd4ce91447ba23b602b645d1f7f/users/40c7c4699490462b97482053134ac70d/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:55.660941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d43fb-4ea7-481f-871c-f1f2a2e48ca9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:55.661280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d43fb-4ea7-481f-871c-f1f2a2e48ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:55.661581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d43fb-4ea7-481f-871c-f1f2a2e48ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:55.704176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-364d43fb-4ea7-481f-871c-f1f2a2e48ca9 None None] MFA Rules not processed for user `40c7c4699490462b97482053134ac70d`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:55.705526 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-364d43fb-4ea7-481f-871c-f1f2a2e48ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:55.726577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab83cd247aae48658343b52330a0f0be', 'request_id': 'req-eac1165f-1e99-4d90-9f1c-6d12e5a88806'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:55.734684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab83cd247aae48658343b52330a0f0be', 'request_id': 'req-eac1165f-1e99-4d90-9f1c-6d12e5a88806'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:55.745942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab83cd247aae48658343b52330a0f0be', 'request_id': 'req-eac1165f-1e99-4d90-9f1c-6d12e5a88806'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:55.762176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] The token cache is being invalidate because project ab83cd247aae48658343b52330a0f0be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:55.762573 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:55.763182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:55.765460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1080/2163] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:55 2022] DELETE /identity/v3/projects/ab83cd247aae48658343b52330a0f0be => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:28:55.770636 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:55.772318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:55.802120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1084/2164] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:55.812388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d43fb-4ea7-481f-871c-f1f2a2e48ca9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:55.814346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-364d43fb-4ea7-481f-871c-f1f2a2e48ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:55.833203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:55.874080 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1165f-1e99-4d90-9f1c-6d12e5a88806 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:55.884075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:55.884075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:55.884075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:55.884075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] PATH_INFO: `/v3/users/bf1f4cb0fe2647839353b7f5958a3596` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:55.885522 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf1f4cb0fe2647839353b7f5958a3596)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:55.887967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:55.914944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d43fb-4ea7-481f-871c-f1f2a2e48ca9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:55.929756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:55.966998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf1f4cb0fe2647839353b7f5958a3596'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:56.003151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf1f4cb0fe2647839353b7f5958a3596'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:56.003866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf1f4cb0fe2647839353b7f5958a3596'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:56.006228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548802d-997d-4f02-802c-ef36fafa235e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:56.007362 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548802d-997d-4f02-802c-ef36fafa235e service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:56.007658 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548802d-997d-4f02-802c-ef36fafa235e service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:56.007934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548802d-997d-4f02-802c-ef36fafa235e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:56.016519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf1f4cb0fe2647839353b7f5958a3596'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:56.018075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf1f4cb0fe2647839353b7f5958a3596'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:56.043378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1081/2165] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:55 2022] DELETE /identity/v3/users/bf1f4cb0fe2647839353b7f5958a3596 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:56.086812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548802d-997d-4f02-802c-ef36fafa235e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:56.088224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548802d-997d-4f02-802c-ef36fafa235e service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:56.100277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1085/2166] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:28:56.368469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:56.369952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:56.461695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fcc85-0079-413c-8996-015f803f75d4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:56.467049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:56.467909 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:56.468200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:56.468465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] PATH_INFO: `/v3/projects/ea1b7c268dc84c31b630d73e071700c3` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:56.469647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea1b7c268dc84c31b630d73e071700c3)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:56.471637 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:56.541519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea1b7c268dc84c31b630d73e071700c3', 'request_id': 'req-1ac2c116-ff90-4832-a29e-c532100b575d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:56.549167 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea1b7c268dc84c31b630d73e071700c3', 'request_id': 'req-1ac2c116-ff90-4832-a29e-c532100b575d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:56.562560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea1b7c268dc84c31b630d73e071700c3', 'request_id': 'req-1ac2c116-ff90-4832-a29e-c532100b575d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:56.564308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:56.582957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] The token cache is being invalidate because project ea1b7c268dc84c31b630d73e071700c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:56.583853 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:56.585254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-1ac2c116-ff90-4832-a29e-c532100b575d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:56.592815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab08fdc-f19d-4f82-8e95-3d3dd38dc1cb None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:56.593170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab08fdc-f19d-4f82-8e95-3d3dd38dc1cb None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:56.593513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab08fdc-f19d-4f82-8e95-3d3dd38dc1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:56.595443 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1082/2167] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:56 2022] DELETE /identity/v3/projects/ea1b7c268dc84c31b630d73e071700c3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:56.640380 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0ab08fdc-f19d-4f82-8e95-3d3dd38dc1cb None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:56.642916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0ab08fdc-f19d-4f82-8e95-3d3dd38dc1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:56.742732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1086/2168] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:56.747013 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe4cd6-5443-4c25-8e8a-08b07b0888dc None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:56.747312 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe4cd6-5443-4c25-8e8a-08b07b0888dc None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:56.747590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe4cd6-5443-4c25-8e8a-08b07b0888dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:56.772123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-78fe4cd6-5443-4c25-8e8a-08b07b0888dc None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:56.773399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-78fe4cd6-5443-4c25-8e8a-08b07b0888dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:56.787326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1083/2169] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:56.789298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab08fdc-f19d-4f82-8e95-3d3dd38dc1cb None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:56.802755 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab08fdc-f19d-4f82-8e95-3d3dd38dc1cb None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:56.810116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1146e98-a79b-4373-8b35-deba4829d8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:56.811023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1146e98-a79b-4373-8b35-deba4829d8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:56.811308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1146e98-a79b-4373-8b35-deba4829d8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:56.812198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1146e98-a79b-4373-8b35-deba4829d8d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:56.812384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1146e98-a79b-4373-8b35-deba4829d8d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:28:56.812694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1146e98-a79b-4373-8b35-deba4829d8d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:56.814360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1146e98-a79b-4373-8b35-deba4829d8d6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:56.832665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1087/2170] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:28:56.833885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe4cd6-5443-4c25-8e8a-08b07b0888dc None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:56.844320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe4cd6-5443-4c25-8e8a-08b07b0888dc None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:56.850245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58097465-41c3-4f39-9a21-1362a0d80870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:56.850956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58097465-41c3-4f39-9a21-1362a0d80870 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:56.851245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58097465-41c3-4f39-9a21-1362a0d80870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:56.851517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58097465-41c3-4f39-9a21-1362a0d80870 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:56.852221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58097465-41c3-4f39-9a21-1362a0d80870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:56.854154 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58097465-41c3-4f39-9a21-1362a0d80870 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:56.885646 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1084/2171] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:56 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:56.889395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1146e98-a79b-4373-8b35-deba4829d8d6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:56.904645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1146e98-a79b-4373-8b35-deba4829d8d6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:56.964878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:56.975306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f50e5e-2da1-4571-975f-8187d4157e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:56.975881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f50e5e-2da1-4571-975f-8187d4157e22 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:56.976158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f50e5e-2da1-4571-975f-8187d4157e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:56.976430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f50e5e-2da1-4571-975f-8187d4157e22 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:56.977121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f50e5e-2da1-4571-975f-8187d4157e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:56.979120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f50e5e-2da1-4571-975f-8187d4157e22 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:57.000004 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1088/2172] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:56 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:57.004021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58097465-41c3-4f39-9a21-1362a0d80870 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:57.015775 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58097465-41c3-4f39-9a21-1362a0d80870 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:57.022934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13feacf5-aae4-4d61-aa66-97d642ad3b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:57.023960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feacf5-aae4-4d61-aa66-97d642ad3b9d admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:57.024286 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feacf5-aae4-4d61-aa66-97d642ad3b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:57.024684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13feacf5-aae4-4d61-aa66-97d642ad3b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:57.025518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13feacf5-aae4-4d61-aa66-97d642ad3b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:57.027477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13feacf5-aae4-4d61-aa66-97d642ad3b9d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:57.051645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1085/2173] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:57.052331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f50e5e-2da1-4571-975f-8187d4157e22 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:57.058224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f50e5e-2da1-4571-975f-8187d4157e22 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:57.060640 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5369e1-2882-4e23-ab07-076958bdcd5a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:57.060930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5369e1-2882-4e23-ab07-076958bdcd5a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:57.061203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5369e1-2882-4e23-ab07-076958bdcd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:57.063059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335ae9e-3508-4579-a701-d96766eb3dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:57.063718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335ae9e-3508-4579-a701-d96766eb3dbd admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:57.063999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335ae9e-3508-4579-a701-d96766eb3dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:57.064300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335ae9e-3508-4579-a701-d96766eb3dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:57.065029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335ae9e-3508-4579-a701-d96766eb3dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:57.066728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335ae9e-3508-4579-a701-d96766eb3dbd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:57.078864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1089/2174] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:57.081914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-cb5369e1-2882-4e23-ab07-076958bdcd5a None None] MFA Rules not processed for user `193a35a91a47410f8b12cb2e84b33e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:57.083263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335ae9e-3508-4579-a701-d96766eb3dbd admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:57.083534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-cb5369e1-2882-4e23-ab07-076958bdcd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:57.093569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335ae9e-3508-4579-a701-d96766eb3dbd admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:57.103276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0bcc2-cd38-469d-8662-ba30c1f1e185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:57.103276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0bcc2-cd38-469d-8662-ba30c1f1e185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:57.105449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0bcc2-cd38-469d-8662-ba30c1f1e185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:57.105449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0bcc2-cd38-469d-8662-ba30c1f1e185 admin admin] PATH_INFO: `/v3/projects/1435a19531ea4462985ff2313b6715b4/users/d05774e363f04beca5990c3b841bdb66/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:57.123671 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0bcc2-cd38-469d-8662-ba30c1f1e185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1435a19531ea4462985ff2313b6715b4, user_id=d05774e363f04beca5990c3b841bdb66, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:57.126941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0bcc2-cd38-469d-8662-ba30c1f1e185 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:57.150457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1090/2175] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:57 2022] PUT /identity/v3/projects/1435a19531ea4462985ff2313b6715b4/users/d05774e363f04beca5990c3b841bdb66/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:28:57.162731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1086/2176] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:57.163347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a26a43-7011-4fe8-abb2-e5712b756b74 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:57.163935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a26a43-7011-4fe8-abb2-e5712b756b74 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:57.164214 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a26a43-7011-4fe8-abb2-e5712b756b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:57.222071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f9a26a43-7011-4fe8-abb2-e5712b756b74 None None] MFA Rules not processed for user `d05774e363f04beca5990c3b841bdb66`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:57.223364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f9a26a43-7011-4fe8-abb2-e5712b756b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:57.351179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1091/2177] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:28:57.355943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5369e1-2882-4e23-ab07-076958bdcd5a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:57.368786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5369e1-2882-4e23-ab07-076958bdcd5a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:57.448409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e9596-625f-47dd-818f-7c03486f22dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:57.449084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9596-625f-47dd-818f-7c03486f22dd admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:57.449364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9596-625f-47dd-818f-7c03486f22dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:57.449633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e9596-625f-47dd-818f-7c03486f22dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:57.450455 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e9596-625f-47dd-818f-7c03486f22dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:57.453639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e9596-625f-47dd-818f-7c03486f22dd admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:57.491054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1087/2178] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:28:57 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:28:57.493882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a26a43-7011-4fe8-abb2-e5712b756b74 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:57.509591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a26a43-7011-4fe8-abb2-e5712b756b74 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:57.621666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfffa8-172e-442f-87b7-394fbcea594d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:57.622068 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfffa8-172e-442f-87b7-394fbcea594d admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:57.622353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfffa8-172e-442f-87b7-394fbcea594d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:57.623295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accfffa8-172e-442f-87b7-394fbcea594d admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:57.623428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accfffa8-172e-442f-87b7-394fbcea594d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:57.625952 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accfffa8-172e-442f-87b7-394fbcea594d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:57.675771 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1092/2179] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:28:57 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:28:57.679540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e9596-625f-47dd-818f-7c03486f22dd admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:57.693181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e9596-625f-47dd-818f-7c03486f22dd admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:57.699541 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2a416-b6ad-41c5-bb87-da97447872b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:57.700444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2a416-b6ad-41c5-bb87-da97447872b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:57.701468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2a416-b6ad-41c5-bb87-da97447872b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:57.701911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2a416-b6ad-41c5-bb87-da97447872b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:57.702916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2a416-b6ad-41c5-bb87-da97447872b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:57.704741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2a416-b6ad-41c5-bb87-da97447872b1 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:57.719801 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1088/2180] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:28:57.724142 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfffa8-172e-442f-87b7-394fbcea594d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:57.739329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accfffa8-172e-442f-87b7-394fbcea594d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:57.746645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218f4f5-c90d-4c00-a76e-af363fd66db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:57.747891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218f4f5-c90d-4c00-a76e-af363fd66db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:57.748321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218f4f5-c90d-4c00-a76e-af363fd66db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:57.748744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218f4f5-c90d-4c00-a76e-af363fd66db2 admin admin] PATH_INFO: `/v3/projects/a9e3f826e2bc4d108695f2e0e8bc4291/users/7f5d08cc52254745928d220e8152487c/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:57.785153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218f4f5-c90d-4c00-a76e-af363fd66db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e3f826e2bc4d108695f2e0e8bc4291, user_id=7f5d08cc52254745928d220e8152487c, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:57.788100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218f4f5-c90d-4c00-a76e-af363fd66db2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:57.809624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2a416-b6ad-41c5-bb87-da97447872b1 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:57.820301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1093/2181] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:57 2022] PUT /identity/v3/projects/a9e3f826e2bc4d108695f2e0e8bc4291/users/7f5d08cc52254745928d220e8152487c/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:57.823759 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2a416-b6ad-41c5-bb87-da97447872b1 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:57.867907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:57.913946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894216e3-f493-4085-b5a9-f9d10c75d242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:57.914730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894216e3-f493-4085-b5a9-f9d10c75d242 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:57.915582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894216e3-f493-4085-b5a9-f9d10c75d242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:57.915931 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894216e3-f493-4085-b5a9-f9d10c75d242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:57.916816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894216e3-f493-4085-b5a9-f9d10c75d242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:57.920297 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894216e3-f493-4085-b5a9-f9d10c75d242 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:57.934971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1089/2182] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:28:57.939896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218f4f5-c90d-4c00-a76e-af363fd66db2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:57.950221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:57.957139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218f4f5-c90d-4c00-a76e-af363fd66db2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:57.973557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260cd71-163f-49a8-9a61-d7cc022e2574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:57.974718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260cd71-163f-49a8-9a61-d7cc022e2574 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:57.975100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260cd71-163f-49a8-9a61-d7cc022e2574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:57.975514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260cd71-163f-49a8-9a61-d7cc022e2574 admin admin] PATH_INFO: `/v3/projects/a9e3f826e2bc4d108695f2e0e8bc4291/users/7f5d08cc52254745928d220e8152487c/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:57.977992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260cd71-163f-49a8-9a61-d7cc022e2574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e3f826e2bc4d108695f2e0e8bc4291, user_id=7f5d08cc52254745928d220e8152487c, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:57.982609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b260cd71-163f-49a8-9a61-d7cc022e2574 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:57.999558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1094/2183] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:28:57 2022] PUT /identity/v3/projects/a9e3f826e2bc4d108695f2e0e8bc4291/users/7f5d08cc52254745928d220e8152487c/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:58.012288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecb4ae-37cf-4ad6-b3d8-0ea9fa8e8464 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:58.012885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecb4ae-37cf-4ad6-b3d8-0ea9fa8e8464 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:58.012885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cecb4ae-37cf-4ad6-b3d8-0ea9fa8e8464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:58.083676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0cecb4ae-37cf-4ad6-b3d8-0ea9fa8e8464 None None] MFA Rules not processed for user `7f5d08cc52254745928d220e8152487c`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:58.085882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0cecb4ae-37cf-4ad6-b3d8-0ea9fa8e8464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:58.188637 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1090/2184] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:28:58.196428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260cd71-163f-49a8-9a61-d7cc022e2574 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:58.198291 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b260cd71-163f-49a8-9a61-d7cc022e2574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:58.274963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260cd71-163f-49a8-9a61-d7cc022e2574 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:58.279839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a250fd2-c7e3-4b32-ae33-3b2e90959e55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:58.280780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a250fd2-c7e3-4b32-ae33-3b2e90959e55 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:58.281201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a250fd2-c7e3-4b32-ae33-3b2e90959e55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:58.281789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a250fd2-c7e3-4b32-ae33-3b2e90959e55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:58.294596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a250fd2-c7e3-4b32-ae33-3b2e90959e55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:58.296817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a250fd2-c7e3-4b32-ae33-3b2e90959e55 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:58.306274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1095/2185] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:58.777980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecb4ae-37cf-4ad6-b3d8-0ea9fa8e8464 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:58.850262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cecb4ae-37cf-4ad6-b3d8-0ea9fa8e8464 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:58.854985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:58.855928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:58.856313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:58.856651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] PATH_INFO: `/v3/users/d05774e363f04beca5990c3b841bdb66` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:58.857910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d05774e363f04beca5990c3b841bdb66)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:58.859974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:58.911434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd05774e363f04beca5990c3b841bdb66'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:58.919341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:58.926249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd05774e363f04beca5990c3b841bdb66'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:58.927001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd05774e363f04beca5990c3b841bdb66'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:58.934743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd05774e363f04beca5990c3b841bdb66'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:58.935405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd05774e363f04beca5990c3b841bdb66'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:58.949177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1091/2186] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:58 2022] DELETE /identity/v3/users/d05774e363f04beca5990c3b841bdb66 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:58.955228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d394a-84e2-4537-b45b-9a523cc68c7f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:58.955520 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d394a-84e2-4537-b45b-9a523cc68c7f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:58.955830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3d394a-84e2-4537-b45b-9a523cc68c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:58.973654 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bb3d394a-84e2-4537-b45b-9a523cc68c7f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:28:58.975076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bb3d394a-84e2-4537-b45b-9a523cc68c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:59.032277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1096/2187] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:28:59.041965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:59.043405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:59.107172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075412a1-456e-4bd3-b053-9feee4baf209 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:59.110923 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662e85b-0612-493c-8545-ec5bf254f52d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:59.111562 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662e85b-0612-493c-8545-ec5bf254f52d service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:59.111842 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662e85b-0612-493c-8545-ec5bf254f52d service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:59.112762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662e85b-0612-493c-8545-ec5bf254f52d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:59.122753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662e85b-0612-493c-8545-ec5bf254f52d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:59.124205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4662e85b-0612-493c-8545-ec5bf254f52d service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:59.135615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1092/2188] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:28:59.547061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d394a-84e2-4537-b45b-9a523cc68c7f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:59.548517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bb3d394a-84e2-4537-b45b-9a523cc68c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:28:59.561430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3d394a-84e2-4537-b45b-9a523cc68c7f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:59.566809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:59.567900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:59.568332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:59.568761 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] PATH_INFO: `/v3/projects/1435a19531ea4462985ff2313b6715b4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:59.570235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1435a19531ea4462985ff2313b6715b4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:59.572268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:59.628551 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1435a19531ea4462985ff2313b6715b4', 'request_id': 'req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:59.635714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1435a19531ea4462985ff2313b6715b4', 'request_id': 'req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:59.645443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1435a19531ea4462985ff2313b6715b4', 'request_id': 'req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:59.658431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] The token cache is being invalidate because project 1435a19531ea4462985ff2313b6715b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:28:59.659842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:59.660079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:59.662368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1097/2189] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:28:59 2022] DELETE /identity/v3/projects/1435a19531ea4462985ff2313b6715b4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:28:59.666308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662e85b-0612-493c-8545-ec5bf254f52d service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:28:59.681552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662e85b-0612-493c-8545-ec5bf254f52d service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:28:59.746849 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70d7e4-01f9-4e43-8866-ef71f5bb81af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:28:59.747718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70d7e4-01f9-4e43-8866-ef71f5bb81af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:28:59.747998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70d7e4-01f9-4e43-8866-ef71f5bb81af admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:28:59.748271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70d7e4-01f9-4e43-8866-ef71f5bb81af admin admin] PATH_INFO: `/v3/users/7f5d08cc52254745928d220e8152487c` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:28:59.749467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70d7e4-01f9-4e43-8866-ef71f5bb81af admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5d08cc52254745928d220e8152487c)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:28:59.752096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70d7e4-01f9-4e43-8866-ef71f5bb81af admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:28:59.823123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5a70d7e4-01f9-4e43-8866-ef71f5bb81af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5d08cc52254745928d220e8152487c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:59.840486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5a70d7e4-01f9-4e43-8866-ef71f5bb81af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5d08cc52254745928d220e8152487c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:59.841606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5a70d7e4-01f9-4e43-8866-ef71f5bb81af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5d08cc52254745928d220e8152487c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:59.866587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5a70d7e4-01f9-4e43-8866-ef71f5bb81af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5d08cc52254745928d220e8152487c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:59.868272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5a70d7e4-01f9-4e43-8866-ef71f5bb81af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5d08cc52254745928d220e8152487c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:28:59.875073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:28:59.891169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1093/2190] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:28:59 2022] DELETE /identity/v3/users/7f5d08cc52254745928d220e8152487c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:29:00.544078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:00.545794 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:00.559463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e33cf-bcc5-4185-a492-b2f7657f36b2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:00.580431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:00.694948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:00.695841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:00.696262 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:00.696928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] PATH_INFO: `/v3/projects/a9e3f826e2bc4d108695f2e0e8bc4291` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:00.698356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e3f826e2bc4d108695f2e0e8bc4291)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:00.700662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:00.806588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e3f826e2bc4d108695f2e0e8bc4291', 'request_id': 'req-19664631-ec24-4fbc-b967-803ce097537a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:00.815910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e3f826e2bc4d108695f2e0e8bc4291', 'request_id': 'req-19664631-ec24-4fbc-b967-803ce097537a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:00.832265 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e3f826e2bc4d108695f2e0e8bc4291', 'request_id': 'req-19664631-ec24-4fbc-b967-803ce097537a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:00.862494 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] The token cache is being invalidate because project a9e3f826e2bc4d108695f2e0e8bc4291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:29:00.862816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:00.863474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-19664631-ec24-4fbc-b967-803ce097537a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:00.865250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1098/2191] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:29:00 2022] DELETE /identity/v3/projects/a9e3f826e2bc4d108695f2e0e8bc4291 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:29:00.869992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34bc8e-1a73-44cb-88de-e86bf6f375fd None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:00.870489 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34bc8e-1a73-44cb-88de-e86bf6f375fd None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:00.870605 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34bc8e-1a73-44cb-88de-e86bf6f375fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:00.892888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-bc34bc8e-1a73-44cb-88de-e86bf6f375fd None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:00.895018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-bc34bc8e-1a73-44cb-88de-e86bf6f375fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:00.975830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1094/2192] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:00.985053 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd25f6-6705-449a-8a9c-fb3e0276e118 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:00.985344 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd25f6-6705-449a-8a9c-fb3e0276e118 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:00.985630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd25f6-6705-449a-8a9c-fb3e0276e118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:01.025039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-08fd25f6-6705-449a-8a9c-fb3e0276e118 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:01.026663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-08fd25f6-6705-449a-8a9c-fb3e0276e118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:01.036156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1099/2193] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:01.040352 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34bc8e-1a73-44cb-88de-e86bf6f375fd None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:01.051413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34bc8e-1a73-44cb-88de-e86bf6f375fd None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:01.057025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb016f9-4046-41df-a1f1-afa972bb60b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:01.057836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb016f9-4046-41df-a1f1-afa972bb60b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:01.058288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb016f9-4046-41df-a1f1-afa972bb60b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:01.058723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb016f9-4046-41df-a1f1-afa972bb60b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:01.059561 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb016f9-4046-41df-a1f1-afa972bb60b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:29:01.059966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb016f9-4046-41df-a1f1-afa972bb60b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:01.061708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb016f9-4046-41df-a1f1-afa972bb60b6 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:01.075289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1095/2194] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:29:01.079517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd25f6-6705-449a-8a9c-fb3e0276e118 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:01.094904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd25f6-6705-449a-8a9c-fb3e0276e118 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:01.101636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9a9c2-70ac-46bb-81c2-14efe39fb848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:01.102311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9a9c2-70ac-46bb-81c2-14efe39fb848 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:01.102640 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9a9c2-70ac-46bb-81c2-14efe39fb848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:01.103107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9a9c2-70ac-46bb-81c2-14efe39fb848 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:01.103835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9a9c2-70ac-46bb-81c2-14efe39fb848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:01.105845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9a9c2-70ac-46bb-81c2-14efe39fb848 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:01.154947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1100/2195] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:29:01 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:29:01.159293 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb016f9-4046-41df-a1f1-afa972bb60b6 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:01.172296 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb016f9-4046-41df-a1f1-afa972bb60b6 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:01.254064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5745c-a920-4136-9d65-adb6a2481d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:01.254930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5745c-a920-4136-9d65-adb6a2481d3b admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:01.255327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5745c-a920-4136-9d65-adb6a2481d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:01.256822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5745c-a920-4136-9d65-adb6a2481d3b admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:01.258213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5745c-a920-4136-9d65-adb6a2481d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:01.266275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5745c-a920-4136-9d65-adb6a2481d3b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:01.307395 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1096/2196] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:29:01 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:29:01.311729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9a9c2-70ac-46bb-81c2-14efe39fb848 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:01.334239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9a9c2-70ac-46bb-81c2-14efe39fb848 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:01.342875 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14bff2-7f26-439d-aea9-8287423fc978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:01.346714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14bff2-7f26-439d-aea9-8287423fc978 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:01.347078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14bff2-7f26-439d-aea9-8287423fc978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:01.347360 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14bff2-7f26-439d-aea9-8287423fc978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:01.348131 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14bff2-7f26-439d-aea9-8287423fc978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:01.354123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14bff2-7f26-439d-aea9-8287423fc978 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:01.375089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1101/2197] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:29:01.377192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5745c-a920-4136-9d65-adb6a2481d3b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:01.390566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5745c-a920-4136-9d65-adb6a2481d3b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:01.399708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b0dee-d042-446f-a015-51c21a4a4a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:01.400914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b0dee-d042-446f-a015-51c21a4a4a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:01.401377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b0dee-d042-446f-a015-51c21a4a4a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:01.401803 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b0dee-d042-446f-a015-51c21a4a4a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:01.402716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b0dee-d042-446f-a015-51c21a4a4a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:01.404503 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b0dee-d042-446f-a015-51c21a4a4a91 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:01.422052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1097/2198] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:29:01.422697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14bff2-7f26-439d-aea9-8287423fc978 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:01.461531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14bff2-7f26-439d-aea9-8287423fc978 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:01.461531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb62a6-9640-45c9-b12b-0fa1d5c00b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:01.463603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb62a6-9640-45c9-b12b-0fa1d5c00b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:01.463881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb62a6-9640-45c9-b12b-0fa1d5c00b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:01.464171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb62a6-9640-45c9-b12b-0fa1d5c00b49 admin admin] PATH_INFO: `/v3/projects/299dfc6bc13e4fd2b34ce37dcaed8e80/users/d19f5f7eda2b49e597ed75ace8d97768/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:01.486220 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb62a6-9640-45c9-b12b-0fa1d5c00b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=299dfc6bc13e4fd2b34ce37dcaed8e80, user_id=d19f5f7eda2b49e597ed75ace8d97768, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:01.489221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb62a6-9640-45c9-b12b-0fa1d5c00b49 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:01.511155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1102/2199] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:29:01 2022] PUT /identity/v3/projects/299dfc6bc13e4fd2b34ce37dcaed8e80/users/d19f5f7eda2b49e597ed75ace8d97768/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:29:01.523354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13620-9642-4056-a305-f2ef1d71e5bc None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:01.523687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13620-9642-4056-a305-f2ef1d71e5bc None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:01.523942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd13620-9642-4056-a305-f2ef1d71e5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:01.565808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-dcd13620-9642-4056-a305-f2ef1d71e5bc None None] MFA Rules not processed for user `d19f5f7eda2b49e597ed75ace8d97768`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:01.567117 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-dcd13620-9642-4056-a305-f2ef1d71e5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:01.670385 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:01.700152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1098/2200] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:29:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:01.712295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb62a6-9640-45c9-b12b-0fa1d5c00b49 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:01.714310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3bdb62a6-9640-45c9-b12b-0fa1d5c00b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:01.809427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:01.835116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb62a6-9640-45c9-b12b-0fa1d5c00b49 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:01.847770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050b525-1d5e-4562-b7c2-9a78f57345b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:01.848795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b525-1d5e-4562-b7c2-9a78f57345b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:01.849314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b525-1d5e-4562-b7c2-9a78f57345b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:01.849682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050b525-1d5e-4562-b7c2-9a78f57345b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:01.860827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050b525-1d5e-4562-b7c2-9a78f57345b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:01.863024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050b525-1d5e-4562-b7c2-9a78f57345b1 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:01.876505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1103/2201] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:29:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:29:01.905984 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd13620-9642-4056-a305-f2ef1d71e5bc None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:02.018919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd13620-9642-4056-a305-f2ef1d71e5bc None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:02.025161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:02.026563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:02.027043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:02.027410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] PATH_INFO: `/v3/users/d19f5f7eda2b49e597ed75ace8d97768` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:02.028751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d19f5f7eda2b49e597ed75ace8d97768)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:02.030815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:02.094157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd19f5f7eda2b49e597ed75ace8d97768'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:02.115643 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd19f5f7eda2b49e597ed75ace8d97768'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:02.116560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd19f5f7eda2b49e597ed75ace8d97768'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:02.127536 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd19f5f7eda2b49e597ed75ace8d97768'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:02.128373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd19f5f7eda2b49e597ed75ace8d97768'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:02.150337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1099/2202] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:29:01 2022] DELETE /identity/v3/users/d19f5f7eda2b49e597ed75ace8d97768 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:29:02.154552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c39faf-c554-4c02-bd7b-fbdda9a3aa44 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:02.154860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c39faf-c554-4c02-bd7b-fbdda9a3aa44 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:02.155195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c39faf-c554-4c02-bd7b-fbdda9a3aa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:02.177161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f6c39faf-c554-4c02-bd7b-fbdda9a3aa44 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:02.179490 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f6c39faf-c554-4c02-bd7b-fbdda9a3aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:02.260379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1104/2203] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:02.271772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:02.273252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:02.340767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c9a1bf-51d7-4b3c-b221-0e1e64e18432 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:02.344690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7181b19a-07db-47b4-b676-3b4af300522c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:02.347410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181b19a-07db-47b4-b676-3b4af300522c service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:02.347695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181b19a-07db-47b4-b676-3b4af300522c service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:02.347978 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7181b19a-07db-47b4-b676-3b4af300522c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:02.359389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7181b19a-07db-47b4-b676-3b4af300522c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:02.361222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7181b19a-07db-47b4-b676-3b4af300522c service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:02.372907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1100/2204] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:29:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:29:02.670943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c39faf-c554-4c02-bd7b-fbdda9a3aa44 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:02.678601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f6c39faf-c554-4c02-bd7b-fbdda9a3aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:02.695185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c39faf-c554-4c02-bd7b-fbdda9a3aa44 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:02.700122 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:02.701185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:02.701606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:02.701974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] PATH_INFO: `/v3/projects/299dfc6bc13e4fd2b34ce37dcaed8e80` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:02.703313 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=299dfc6bc13e4fd2b34ce37dcaed8e80)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:02.705376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:02.784607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '299dfc6bc13e4fd2b34ce37dcaed8e80', 'request_id': 'req-68909f09-48a7-4149-93d5-e96f3dabd891'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:02.791679 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '299dfc6bc13e4fd2b34ce37dcaed8e80', 'request_id': 'req-68909f09-48a7-4149-93d5-e96f3dabd891'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:02.819816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '299dfc6bc13e4fd2b34ce37dcaed8e80', 'request_id': 'req-68909f09-48a7-4149-93d5-e96f3dabd891'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:02.833862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] The token cache is being invalidate because project 299dfc6bc13e4fd2b34ce37dcaed8e80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:29:02.834619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:02.835336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-68909f09-48a7-4149-93d5-e96f3dabd891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:02.842794 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1105/2205] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:29:02 2022] DELETE /identity/v3/projects/299dfc6bc13e4fd2b34ce37dcaed8e80 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:29:02.844861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22989e2-ae2a-4e5c-a0d7-251a69990f0c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:02.845290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22989e2-ae2a-4e5c-a0d7-251a69990f0c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:02.845691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22989e2-ae2a-4e5c-a0d7-251a69990f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:02.873698 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-a22989e2-ae2a-4e5c-a0d7-251a69990f0c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:02.875296 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a22989e2-ae2a-4e5c-a0d7-251a69990f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:02.888469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:02.948334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1101/2206] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:02.956642 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece46dc-4c99-4ba1-8c00-ec856fc96ae4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:02.956935 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece46dc-4c99-4ba1-8c00-ec856fc96ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:02.957220 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece46dc-4c99-4ba1-8c00-ec856fc96ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:02.975996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5ece46dc-4c99-4ba1-8c00-ec856fc96ae4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:02.977208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5ece46dc-4c99-4ba1-8c00-ec856fc96ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:02.986156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1106/2207] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:02.991585 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22989e2-ae2a-4e5c-a0d7-251a69990f0c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:03.001736 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22989e2-ae2a-4e5c-a0d7-251a69990f0c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:03.009591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672b03f-1605-4d78-97e6-5a56aaf1bf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:03.010330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672b03f-1605-4d78-97e6-5a56aaf1bf96 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:03.010708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672b03f-1605-4d78-97e6-5a56aaf1bf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:03.011001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0672b03f-1605-4d78-97e6-5a56aaf1bf96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:03.017279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672b03f-1605-4d78-97e6-5a56aaf1bf96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:29:03.017279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672b03f-1605-4d78-97e6-5a56aaf1bf96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:03.017979 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0672b03f-1605-4d78-97e6-5a56aaf1bf96 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:03.033922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1102/2208] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:29:03.038322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece46dc-4c99-4ba1-8c00-ec856fc96ae4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:03.049358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece46dc-4c99-4ba1-8c00-ec856fc96ae4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:03.057536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c285ca-3e6f-4aae-b12b-2aa1519b972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:03.058361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c285ca-3e6f-4aae-b12b-2aa1519b972c admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:03.058778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c285ca-3e6f-4aae-b12b-2aa1519b972c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:03.059319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c285ca-3e6f-4aae-b12b-2aa1519b972c admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:03.060111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c285ca-3e6f-4aae-b12b-2aa1519b972c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:03.062201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c285ca-3e6f-4aae-b12b-2aa1519b972c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:03.100818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1107/2209] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:29:03 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:29:03.105578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672b03f-1605-4d78-97e6-5a56aaf1bf96 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:03.127226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0672b03f-1605-4d78-97e6-5a56aaf1bf96 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:03.216340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da074e-b041-4cf3-8892-24e4a25e25a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:03.217095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da074e-b041-4cf3-8892-24e4a25e25a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:03.217449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da074e-b041-4cf3-8892-24e4a25e25a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:03.217797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da074e-b041-4cf3-8892-24e4a25e25a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:03.218595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da074e-b041-4cf3-8892-24e4a25e25a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:03.220797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da074e-b041-4cf3-8892-24e4a25e25a0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:03.242023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1103/2210] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:29:03 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:29:03.246087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c285ca-3e6f-4aae-b12b-2aa1519b972c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:03.257357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c285ca-3e6f-4aae-b12b-2aa1519b972c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:03.262079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7c2d4-9f88-4a3a-a3ee-98fecff3a046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:03.262868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c2d4-9f88-4a3a-a3ee-98fecff3a046 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:03.263222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c2d4-9f88-4a3a-a3ee-98fecff3a046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:03.263566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c2d4-9f88-4a3a-a3ee-98fecff3a046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:03.264368 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7c2d4-9f88-4a3a-a3ee-98fecff3a046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:03.266036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e7c2d4-9f88-4a3a-a3ee-98fecff3a046 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:03.280656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1108/2211] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:29:03.284496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da074e-b041-4cf3-8892-24e4a25e25a0 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:03.294638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da074e-b041-4cf3-8892-24e4a25e25a0 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:03.299904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0cb30-e3c6-4203-9b88-53a666fd05ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:03.300568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0cb30-e3c6-4203-9b88-53a666fd05ea admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:03.300857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0cb30-e3c6-4203-9b88-53a666fd05ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:03.301140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0cb30-e3c6-4203-9b88-53a666fd05ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:03.301873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0cb30-e3c6-4203-9b88-53a666fd05ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:03.303538 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0cb30-e3c6-4203-9b88-53a666fd05ea admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:03.314199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1104/2212] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:29:03.318947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7c2d4-9f88-4a3a-a3ee-98fecff3a046 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:03.329448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7c2d4-9f88-4a3a-a3ee-98fecff3a046 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:03.334237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72091e46-f781-4f11-811a-ed12ed8bd247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:03.335171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72091e46-f781-4f11-811a-ed12ed8bd247 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:03.335460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72091e46-f781-4f11-811a-ed12ed8bd247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:03.335740 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72091e46-f781-4f11-811a-ed12ed8bd247 admin admin] PATH_INFO: `/v3/projects/5a5fdde934b44fc1b5f79272a938a213/users/a64d4704d19540c78222d40a1ec6c79d/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:03.350525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72091e46-f781-4f11-811a-ed12ed8bd247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a5fdde934b44fc1b5f79272a938a213, user_id=a64d4704d19540c78222d40a1ec6c79d, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:03.353267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72091e46-f781-4f11-811a-ed12ed8bd247 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:03.367819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1109/2213] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:29:03 2022] PUT /identity/v3/projects/5a5fdde934b44fc1b5f79272a938a213/users/a64d4704d19540c78222d40a1ec6c79d/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:29:03.435213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bfa2-be5d-47cc-b58a-dc689903b6c9 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:03.435569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bfa2-be5d-47cc-b58a-dc689903b6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:03.435914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67bfa2-be5d-47cc-b58a-dc689903b6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:03.464485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-db67bfa2-be5d-47cc-b58a-dc689903b6c9 None None] MFA Rules not processed for user `a64d4704d19540c78222d40a1ec6c79d`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:03.466147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-db67bfa2-be5d-47cc-b58a-dc689903b6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:03.546722 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1105/2214] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:29:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:03.558473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72091e46-f781-4f11-811a-ed12ed8bd247 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:03.560378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-72091e46-f781-4f11-811a-ed12ed8bd247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:03.571303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:03.621234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72091e46-f781-4f11-811a-ed12ed8bd247 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:03.624882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b2fa4-de7c-48ef-ab45-d4e77c95dd89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:03.625498 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b2fa4-de7c-48ef-ab45-d4e77c95dd89 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:03.626059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b2fa4-de7c-48ef-ab45-d4e77c95dd89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:03.626339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b2fa4-de7c-48ef-ab45-d4e77c95dd89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:03.635785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b2fa4-de7c-48ef-ab45-d4e77c95dd89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:03.637243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b2fa4-de7c-48ef-ab45-d4e77c95dd89 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:03.646647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1110/2215] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:29:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:29:06.245797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67bfa2-be5d-47cc-b58a-dc689903b6c9 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:06.261006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:06.363265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67bfa2-be5d-47cc-b58a-dc689903b6c9 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:06.371460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:06.372482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:06.372763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:06.373037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] PATH_INFO: `/v3/users/a64d4704d19540c78222d40a1ec6c79d` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:06.374716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a64d4704d19540c78222d40a1ec6c79d)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:06.378748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:06.439767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a64d4704d19540c78222d40a1ec6c79d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:06.447162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a64d4704d19540c78222d40a1ec6c79d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:06.447845 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a64d4704d19540c78222d40a1ec6c79d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:06.455242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a64d4704d19540c78222d40a1ec6c79d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:06.456487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a64d4704d19540c78222d40a1ec6c79d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:06.469909 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1106/2216] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:29:06 2022] DELETE /identity/v3/users/a64d4704d19540c78222d40a1ec6c79d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:29:06.475747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64884f1-2ab7-46f3-9b22-6a70d9ca56cc None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:06.476240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64884f1-2ab7-46f3-9b22-6a70d9ca56cc None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:06.476425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64884f1-2ab7-46f3-9b22-6a70d9ca56cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:06.494510 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c64884f1-2ab7-46f3-9b22-6a70d9ca56cc None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:06.495770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c64884f1-2ab7-46f3-9b22-6a70d9ca56cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:06.561995 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1111/2217] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:06.572228 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:06.573675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:06.637202 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc7797-44a0-4162-9c4e-dfcf60ef4df5 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:06.641254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21b39c-2c66-4bba-b6bc-e774e6191891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:06.641907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21b39c-2c66-4bba-b6bc-e774e6191891 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:06.642188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21b39c-2c66-4bba-b6bc-e774e6191891 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:06.642458 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21b39c-2c66-4bba-b6bc-e774e6191891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:06.653398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21b39c-2c66-4bba-b6bc-e774e6191891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:06.654401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21b39c-2c66-4bba-b6bc-e774e6191891 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:06.666022 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1107/2218] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:29:06.879206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64884f1-2ab7-46f3-9b22-6a70d9ca56cc None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:06.880731 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c64884f1-2ab7-46f3-9b22-6a70d9ca56cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:06.893274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64884f1-2ab7-46f3-9b22-6a70d9ca56cc None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:06.898783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5ae241-23fb-406d-9795-9546a3abfc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:06.899586 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ae241-23fb-406d-9795-9546a3abfc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:06.899954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ae241-23fb-406d-9795-9546a3abfc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:06.900306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5ae241-23fb-406d-9795-9546a3abfc4a admin admin] PATH_INFO: `/v3/users/40c7c4699490462b97482053134ac70d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:06.901976 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ae241-23fb-406d-9795-9546a3abfc4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c7c4699490462b97482053134ac70d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:06.903694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5ae241-23fb-406d-9795-9546a3abfc4a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:06.970866 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac5ae241-23fb-406d-9795-9546a3abfc4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c7c4699490462b97482053134ac70d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:06.985974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21b39c-2c66-4bba-b6bc-e774e6191891 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:06.987894 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2a21b39c-2c66-4bba-b6bc-e774e6191891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:06.989930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac5ae241-23fb-406d-9795-9546a3abfc4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c7c4699490462b97482053134ac70d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:06.991195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac5ae241-23fb-406d-9795-9546a3abfc4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c7c4699490462b97482053134ac70d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:06.999763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac5ae241-23fb-406d-9795-9546a3abfc4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c7c4699490462b97482053134ac70d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.000456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ac5ae241-23fb-406d-9795-9546a3abfc4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c7c4699490462b97482053134ac70d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.016868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1112/2219] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:29:06 2022] DELETE /identity/v3/users/40c7c4699490462b97482053134ac70d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:29:07.022105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354395f-6e3b-49c8-a418-49cf5c73e600 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:07.022416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354395f-6e3b-49c8-a418-49cf5c73e600 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:07.022958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354395f-6e3b-49c8-a418-49cf5c73e600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:07.024631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21b39c-2c66-4bba-b6bc-e774e6191891 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:07.042459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c354395f-6e3b-49c8-a418-49cf5c73e600 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:07.044114 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c354395f-6e3b-49c8-a418-49cf5c73e600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:07.062386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:07.072893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:07.108453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:07.109324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:07.109613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:07.109888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] PATH_INFO: `/v3/projects/5a5fdde934b44fc1b5f79272a938a213` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:07.111163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a5fdde934b44fc1b5f79272a938a213)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:07.113251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:07.121999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1113/2220] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:07.128226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354395f-6e3b-49c8-a418-49cf5c73e600 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:07.130587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c354395f-6e3b-49c8-a418-49cf5c73e600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:07.171423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a5fdde934b44fc1b5f79272a938a213', 'request_id': 'req-54382057-92a9-4cc2-a616-fbcf543ee5e6'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.179296 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a5fdde934b44fc1b5f79272a938a213', 'request_id': 'req-54382057-92a9-4cc2-a616-fbcf543ee5e6'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.188490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a5fdde934b44fc1b5f79272a938a213', 'request_id': 'req-54382057-92a9-4cc2-a616-fbcf543ee5e6'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.204878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354395f-6e3b-49c8-a418-49cf5c73e600 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:07.205176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] The token cache is being invalidate because project 5a5fdde934b44fc1b5f79272a938a213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:29:07.206342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.207558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.212714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1108/2221] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:29:06 2022] DELETE /identity/v3/projects/5a5fdde934b44fc1b5f79272a938a213 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:29:07.274111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c9960-e446-4d61-bbd9-3312256ab9c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:07.275529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9960-e446-4d61-bbd9-3312256ab9c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:07.276108 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9960-e446-4d61-bbd9-3312256ab9c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:07.277212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c9960-e446-4d61-bbd9-3312256ab9c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:07.280221 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a65c9960-e446-4d61-bbd9-3312256ab9c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:07.368303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c9960-e446-4d61-bbd9-3312256ab9c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:07.369895 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c9960-e446-4d61-bbd9-3312256ab9c3 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:07.392014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1114/2222] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:29:07.834960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:07.838180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:07.860915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54382057-92a9-4cc2-a616-fbcf543ee5e6 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:07.867869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:07.881915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:07.882910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:07.883145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] PATH_INFO: `/v3/projects/fca5bfd4ce91447ba23b602b645d1f7f` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:07.886353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca5bfd4ce91447ba23b602b645d1f7f)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:07.889060 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:07.951992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca5bfd4ce91447ba23b602b645d1f7f', 'request_id': 'req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.962109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca5bfd4ce91447ba23b602b645d1f7f', 'request_id': 'req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.971938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca5bfd4ce91447ba23b602b645d1f7f', 'request_id': 'req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.986553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] The token cache is being invalidate because project fca5bfd4ce91447ba23b602b645d1f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:29:07.987587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.988883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:29:07.990635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1109/2223] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:29:07 2022] DELETE /identity/v3/projects/fca5bfd4ce91447ba23b602b645d1f7f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:29:08.778303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69319b-2f07-4642-b16e-a5cd952f5810 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:08.778970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69319b-2f07-4642-b16e-a5cd952f5810 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:08.778970 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69319b-2f07-4642-b16e-a5cd952f5810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:08.797842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9a69319b-2f07-4642-b16e-a5cd952f5810 None None] MFA Rules not processed for user `127a6938bc5c4416a9a502d9064344cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:08.799858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9a69319b-2f07-4642-b16e-a5cd952f5810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:08.831974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:08.872014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1115/2224] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:29:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:10.243092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:10.244702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:10.292899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:10.312079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce45af9-710a-4f77-bb9a-76c1e0fc4485 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:10.318156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374c65f-25ee-48b6-a582-7b4fedfb71c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:10.319192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374c65f-25ee-48b6-a582-7b4fedfb71c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:10.319192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374c65f-25ee-48b6-a582-7b4fedfb71c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:10.319192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374c65f-25ee-48b6-a582-7b4fedfb71c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:10.321247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d374c65f-25ee-48b6-a582-7b4fedfb71c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:10.398560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374c65f-25ee-48b6-a582-7b4fedfb71c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:10.400109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374c65f-25ee-48b6-a582-7b4fedfb71c6 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:10.413989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1110/2225] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:29:10 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:29:16.287337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d992e4-44b6-4cdb-965d-57ad30ac5118 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:16.287967 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d992e4-44b6-4cdb-965d-57ad30ac5118 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:16.288371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d992e4-44b6-4cdb-965d-57ad30ac5118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:16.317064 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-84d992e4-44b6-4cdb-965d-57ad30ac5118 None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:16.318384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-84d992e4-44b6-4cdb-965d-57ad30ac5118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:16.383741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1116/2226] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:29:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:16.429255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374c65f-25ee-48b6-a582-7b4fedfb71c6 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:16.431245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d374c65f-25ee-48b6-a582-7b4fedfb71c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:16.515719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374c65f-25ee-48b6-a582-7b4fedfb71c6 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:16.525847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489df257-7b25-43e5-ab9b-75de3a7d246d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:16.526519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489df257-7b25-43e5-ab9b-75de3a7d246d service swift] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:16.527442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489df257-7b25-43e5-ab9b-75de3a7d246d service swift] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:16.527442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489df257-7b25-43e5-ab9b-75de3a7d246d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:16.540966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489df257-7b25-43e5-ab9b-75de3a7d246d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:16.542263 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489df257-7b25-43e5-ab9b-75de3a7d246d service swift] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:16.573175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1111/2227] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:29:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:29:26.376580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=44 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:26.547812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=62 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:32.195957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44208cb-f4c4-4280-9528-03050930ad04 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:32.196540 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44208cb-f4c4-4280-9528-03050930ad04 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:32.196641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44208cb-f4c4-4280-9528-03050930ad04 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:32.198069 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1117/2228] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:29:32.208151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221f29c-6c2a-433f-afd9-9e7baa30f3b3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:32.208691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221f29c-6c2a-433f-afd9-9e7baa30f3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:32.208691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221f29c-6c2a-433f-afd9-9e7baa30f3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:32.261703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-b221f29c-6c2a-433f-afd9-9e7baa30f3b3 None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:32.263110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b221f29c-6c2a-433f-afd9-9e7baa30f3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:32.343877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1112/2229] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:29:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:32.356299 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44208cb-f4c4-4280-9528-03050930ad04 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:32.357990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a44208cb-f4c4-4280-9528-03050930ad04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:32.437177 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44208cb-f4c4-4280-9528-03050930ad04 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:32.440694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06059b-5f49-44f5-9245-6708cc16e3a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13121a263c31433dbd4a5082b4bf3bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:32.441331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06059b-5f49-44f5-9245-6708cc16e3a5 service glance] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:32.441611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06059b-5f49-44f5-9245-6708cc16e3a5 service glance] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:32.442250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06059b-5f49-44f5-9245-6708cc16e3a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:32.451788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06059b-5f49-44f5-9245-6708cc16e3a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:32.453328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06059b-5f49-44f5-9245-6708cc16e3a5 service glance] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:32.464341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1118/2230] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:29:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:29:32.530334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9776d-8a21-48da-8412-eb7781d2a6ce None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:32.530672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9776d-8a21-48da-8412-eb7781d2a6ce None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:32.530950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9776d-8a21-48da-8412-eb7781d2a6ce None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:32.532365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1113/2231] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:29:32.538068 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caa62a-dff9-4b5b-b531-d224ba06c5d2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:32.538356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caa62a-dff9-4b5b-b531-d224ba06c5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:32.538687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caa62a-dff9-4b5b-b531-d224ba06c5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:32.556044 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-38caa62a-dff9-4b5b-b531-d224ba06c5d2 None None] MFA Rules not processed for user `e3f887d5bbcd424884537b95affd96f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:32.557267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-38caa62a-dff9-4b5b-b531-d224ba06c5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:32.563993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1119/2232] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:29:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:32.575665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9776d-8a21-48da-8412-eb7781d2a6ce None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:32.587159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9776d-8a21-48da-8412-eb7781d2a6ce None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:32.590645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f54da4-5bb8-45fe-9b89-1c086a62a1c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:32.591273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f54da4-5bb8-45fe-9b89-1c086a62a1c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:32.591552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f54da4-5bb8-45fe-9b89-1c086a62a1c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:32.591822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f54da4-5bb8-45fe-9b89-1c086a62a1c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:32.599544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f54da4-5bb8-45fe-9b89-1c086a62a1c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:32.601056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f54da4-5bb8-45fe-9b89-1c086a62a1c6 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:32.608758 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1114/2233] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:29:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:29:32.690144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f292d03-ed84-4ddc-90cf-98bf3b831ae6 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:32.690432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f292d03-ed84-4ddc-90cf-98bf3b831ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:32.690738 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f292d03-ed84-4ddc-90cf-98bf3b831ae6 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:32.692199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1120/2234] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:29:32.697247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a94ee-2faf-44c2-a35a-658f8d713efb None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:32.697532 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a94ee-2faf-44c2-a35a-658f8d713efb None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:32.697805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a94ee-2faf-44c2-a35a-658f8d713efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:32.717011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-900a94ee-2faf-44c2-a35a-658f8d713efb None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:32.718240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-900a94ee-2faf-44c2-a35a-658f8d713efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:32.728244 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1115/2235] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:29:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:32.738382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f292d03-ed84-4ddc-90cf-98bf3b831ae6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:32.749182 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f292d03-ed84-4ddc-90cf-98bf3b831ae6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:32.752680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b298a-98f0-44ed-9d2f-50dfcb6585f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:32.753303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b298a-98f0-44ed-9d2f-50dfcb6585f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:32.753581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b298a-98f0-44ed-9d2f-50dfcb6585f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:32.753852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514b298a-98f0-44ed-9d2f-50dfcb6585f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:32.763772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b298a-98f0-44ed-9d2f-50dfcb6585f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:32.765200 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514b298a-98f0-44ed-9d2f-50dfcb6585f5 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:32.776561 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1121/2236] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:29:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:29:36.358581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a973023-cb29-4108-a45e-5ba86f6a2630 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:36.359079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a973023-cb29-4108-a45e-5ba86f6a2630 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:36.359176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a973023-cb29-4108-a45e-5ba86f6a2630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:36.414204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-4a973023-cb29-4108-a45e-5ba86f6a2630 None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:29:36.415465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-4a973023-cb29-4108-a45e-5ba86f6a2630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:29:36.433315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1116/2237] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:29:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:29:36.446401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b298a-98f0-44ed-9d2f-50dfcb6585f5 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:29:36.475479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514b298a-98f0-44ed-9d2f-50dfcb6585f5 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:29:36.492670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14226785-dd2c-4f51-97b8-703f95fef734 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:29:36.493613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14226785-dd2c-4f51-97b8-703f95fef734 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:29:36.493613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14226785-dd2c-4f51-97b8-703f95fef734 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:29:36.493885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14226785-dd2c-4f51-97b8-703f95fef734 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:29:36.511566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14226785-dd2c-4f51-97b8-703f95fef734 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:29:36.513256 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14226785-dd2c-4f51-97b8-703f95fef734 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:29:36.523077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1122/2238] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:29:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:29:46.430496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=73 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:29:46.519491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=54 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:30:09.705455 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2620b2-f430-4534-b7c1-e80af95d4005 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:30:09.705933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2620b2-f430-4534-b7c1-e80af95d4005 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:30:09.706025 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2620b2-f430-4534-b7c1-e80af95d4005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:30:09.723476 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-ae2620b2-f430-4534-b7c1-e80af95d4005 None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:30:09.724689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ae2620b2-f430-4534-b7c1-e80af95d4005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:30:09.731814 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1117/2239] 10.222.0.93 () {58 vars in 967 bytes} [Fri Jun 3 15:30:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:30:09.744808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14226785-dd2c-4f51-97b8-703f95fef734 service swift] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:30:09.755819 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14226785-dd2c-4f51-97b8-703f95fef734 service swift] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:30:09.760825 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c5d34-bcad-40ed-b351-54c6434b3d01 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:30:09.761463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c5d34-bcad-40ed-b351-54c6434b3d01 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:30:09.761746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c5d34-bcad-40ed-b351-54c6434b3d01 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:30:09.762024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c5d34-bcad-40ed-b351-54c6434b3d01 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:30:09.771417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c5d34-bcad-40ed-b351-54c6434b3d01 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:30:09.773033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c5d34-bcad-40ed-b351-54c6434b3d01 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:30:09.781422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1123/2240] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:30:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:30:19.718246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:30:19.777942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=6 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:30:24.530789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2620b2-f430-4534-b7c1-e80af95d4005 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:30:24.532786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ae2620b2-f430-4534-b7c1-e80af95d4005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:30:24.692525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2620b2-f430-4534-b7c1-e80af95d4005 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:30:24.699259 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43796b5-1c5c-43bb-8956-ba0a32fce54a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:30:24.700460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43796b5-1c5c-43bb-8956-ba0a32fce54a service placement] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:30:24.701043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43796b5-1c5c-43bb-8956-ba0a32fce54a service placement] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:30:24.701731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43796b5-1c5c-43bb-8956-ba0a32fce54a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:30:24.706089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b43796b5-1c5c-43bb-8956-ba0a32fce54a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:30:24.744079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43796b5-1c5c-43bb-8956-ba0a32fce54a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:30:24.747492 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43796b5-1c5c-43bb-8956-ba0a32fce54a service placement] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:30:24.769230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1118/2241] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:30:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:30:34.759697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=41 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:30:35.408800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58820bb0-0051-4362-a18c-c7a2fe3d904d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:30:35.409237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58820bb0-0051-4362-a18c-c7a2fe3d904d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:30:35.409444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58820bb0-0051-4362-a18c-c7a2fe3d904d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:30:35.430923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-58820bb0-0051-4362-a18c-c7a2fe3d904d None None] MFA Rules not processed for user `193a35a91a47410f8b12cb2e84b33e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:30:35.432628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-58820bb0-0051-4362-a18c-c7a2fe3d904d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:30:35.488810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1124/2242] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:30:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:30:45.481253 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=25 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:31:06.050618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9ccc-09c0-49bf-b22f-df7f1bb82b2c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:06.051197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9ccc-09c0-49bf-b22f-df7f1bb82b2c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:06.051197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9ccc-09c0-49bf-b22f-df7f1bb82b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:06.070103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0ffe9ccc-09c0-49bf-b22f-df7f1bb82b2c None None] MFA Rules not processed for user `127a6938bc5c4416a9a502d9064344cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:06.071358 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0ffe9ccc-09c0-49bf-b22f-df7f1bb82b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:06.078267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1119/2243] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:31:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:31:10.577495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721cd19-d6be-44ef-86f0-85285c584f70 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:10.578055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721cd19-d6be-44ef-86f0-85285c584f70 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:10.578140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721cd19-d6be-44ef-86f0-85285c584f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:10.597373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7721cd19-d6be-44ef-86f0-85285c584f70 None None] MFA Rules not processed for user `47d7587d10b3487688ed251a85b83c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:10.598627 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7721cd19-d6be-44ef-86f0-85285c584f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:10.671693 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1125/2244] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:31:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:31:16.064992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:31:16.880954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284be5c-3fac-47c9-8186-193a53f395ef None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:16.881245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284be5c-3fac-47c9-8186-193a53f395ef None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:16.881524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5284be5c-3fac-47c9-8186-193a53f395ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:16.901103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-5284be5c-3fac-47c9-8186-193a53f395ef None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:16.902306 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5284be5c-3fac-47c9-8186-193a53f395ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:16.909861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1120/2245] 10.222.0.93 () {58 vars in 967 bytes} [Fri Jun 3 15:31:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:31:16.977589 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721cd19-d6be-44ef-86f0-85285c584f70 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:16.989785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721cd19-d6be-44ef-86f0-85285c584f70 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:16.994796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67230669-fe62-4d29-be50-c7edb8874248 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:16.995440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230669-fe62-4d29-be50-c7edb8874248 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:16.996016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230669-fe62-4d29-be50-c7edb8874248 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:16.996297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230669-fe62-4d29-be50-c7edb8874248 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:17.005230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67230669-fe62-4d29-be50-c7edb8874248 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:17.006889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67230669-fe62-4d29-be50-c7edb8874248 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:17.015016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1126/2246] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:31:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:31:24.884018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284be5c-3fac-47c9-8186-193a53f395ef None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:24.885473 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5284be5c-3fac-47c9-8186-193a53f395ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:24.950482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5284be5c-3fac-47c9-8186-193a53f395ef None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:24.955445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:24.956310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:24.956589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:24.956860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] PATH_INFO: `/v3/users/dac7f01263374a2cbdd892a2513f5ecd` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:24.958049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac7f01263374a2cbdd892a2513f5ecd)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:24.960106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:25.012464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac7f01263374a2cbdd892a2513f5ecd'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.027543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac7f01263374a2cbdd892a2513f5ecd'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.028359 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac7f01263374a2cbdd892a2513f5ecd'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.036535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac7f01263374a2cbdd892a2513f5ecd'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.039438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac7f01263374a2cbdd892a2513f5ecd'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.055094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1121/2247] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:31:24 2022] DELETE /identity/v3/users/dac7f01263374a2cbdd892a2513f5ecd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:31:25.251175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67230669-fe62-4d29-be50-c7edb8874248 service swift] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:25.262271 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67230669-fe62-4d29-be50-c7edb8874248 service swift] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:25.320864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:25.321468 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:25.321749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:25.322022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] PATH_INFO: `/v3/projects/00d2022cc4be4779af1edf1520c439ae` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:25.323233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00d2022cc4be4779af1edf1520c439ae)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:25.325176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:25.378181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00d2022cc4be4779af1edf1520c439ae', 'request_id': 'req-e8b9b58c-e664-423d-a9b1-e7b6150edb77'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.384664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00d2022cc4be4779af1edf1520c439ae', 'request_id': 'req-e8b9b58c-e664-423d-a9b1-e7b6150edb77'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.394256 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00d2022cc4be4779af1edf1520c439ae', 'request_id': 'req-e8b9b58c-e664-423d-a9b1-e7b6150edb77'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.407158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] The token cache is being invalidate because project 00d2022cc4be4779af1edf1520c439ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:31:25.407503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.408458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.410041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1127/2248] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:31:25 2022] DELETE /identity/v3/projects/00d2022cc4be4779af1edf1520c439ae => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:31:25.414572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:25.415946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:25.430427 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5450605-b8a6-4060-9d19-939393f44094 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:25.500546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dec2d1-4755-4ad4-b5dd-6f99d8ec0bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:25.501305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dec2d1-4755-4ad4-b5dd-6f99d8ec0bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:25.501586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dec2d1-4755-4ad4-b5dd-6f99d8ec0bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:25.501860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dec2d1-4755-4ad4-b5dd-6f99d8ec0bf6 admin admin] PATH_INFO: `/v3/users/b876d4f582d0478093e0bb22fbf07ecf` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:25.503151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dec2d1-4755-4ad4-b5dd-6f99d8ec0bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b876d4f582d0478093e0bb22fbf07ecf)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:25.505098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dec2d1-4755-4ad4-b5dd-6f99d8ec0bf6 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:25.522051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:31:25.571554 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-86dec2d1-4755-4ad4-b5dd-6f99d8ec0bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b876d4f582d0478093e0bb22fbf07ecf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.586691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-86dec2d1-4755-4ad4-b5dd-6f99d8ec0bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b876d4f582d0478093e0bb22fbf07ecf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.587915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-86dec2d1-4755-4ad4-b5dd-6f99d8ec0bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b876d4f582d0478093e0bb22fbf07ecf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.597090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-86dec2d1-4755-4ad4-b5dd-6f99d8ec0bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b876d4f582d0478093e0bb22fbf07ecf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.597809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-86dec2d1-4755-4ad4-b5dd-6f99d8ec0bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b876d4f582d0478093e0bb22fbf07ecf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:25.613169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1122/2249] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:31:25 2022] DELETE /identity/v3/users/b876d4f582d0478093e0bb22fbf07ecf => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:31:25.805628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:25.808189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:25.847130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b9b58c-e664-423d-a9b1-e7b6150edb77 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:25.870584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:31:25.924600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:25.925477 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:25.925757 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:25.926029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] PATH_INFO: `/v3/projects/433937ccb338451d8b27f5be7c1d78fa` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:25.927276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=433937ccb338451d8b27f5be7c1d78fa)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:25.929287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:25.996684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '433937ccb338451d8b27f5be7c1d78fa', 'request_id': 'req-1b0ce7b0-2054-4f5a-a5b9-9977257da357'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:26.003520 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '433937ccb338451d8b27f5be7c1d78fa', 'request_id': 'req-1b0ce7b0-2054-4f5a-a5b9-9977257da357'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:26.014063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '433937ccb338451d8b27f5be7c1d78fa', 'request_id': 'req-1b0ce7b0-2054-4f5a-a5b9-9977257da357'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:26.031187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] The token cache is being invalidate because project 433937ccb338451d8b27f5be7c1d78fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:31:26.031187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:26.031991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1b0ce7b0-2054-4f5a-a5b9-9977257da357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:31:26.033571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1128/2250] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:31:25 2022] DELETE /identity/v3/projects/433937ccb338451d8b27f5be7c1d78fa => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:31:26.039210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb552eb-66f3-4cb7-934a-6b5e8a9af07e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:26.040613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb552eb-66f3-4cb7-934a-6b5e8a9af07e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:26.040613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb552eb-66f3-4cb7-934a-6b5e8a9af07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:26.063308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-9bb552eb-66f3-4cb7-934a-6b5e8a9af07e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:26.064534 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9bb552eb-66f3-4cb7-934a-6b5e8a9af07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:26.131670 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1123/2251] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:31:26.139890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2c0b4-3a98-411d-9e36-837ee0463eb2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:26.140431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2c0b4-3a98-411d-9e36-837ee0463eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:26.141460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2c0b4-3a98-411d-9e36-837ee0463eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:26.162080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bea2c0b4-3a98-411d-9e36-837ee0463eb2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:26.163316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bea2c0b4-3a98-411d-9e36-837ee0463eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:26.171901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1129/2252] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:31:26.176399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb552eb-66f3-4cb7-934a-6b5e8a9af07e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:26.188222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb552eb-66f3-4cb7-934a-6b5e8a9af07e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:26.194335 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb45b5c-cf9d-44a0-ade7-0876d800af15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:26.195032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45b5c-cf9d-44a0-ade7-0876d800af15 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:26.195309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45b5c-cf9d-44a0-ade7-0876d800af15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:26.195583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb45b5c-cf9d-44a0-ade7-0876d800af15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:26.196331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb45b5c-cf9d-44a0-ade7-0876d800af15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:31:26.196618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb45b5c-cf9d-44a0-ade7-0876d800af15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:26.198223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb45b5c-cf9d-44a0-ade7-0876d800af15 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:26.211485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1124/2253] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:31:26.215981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2c0b4-3a98-411d-9e36-837ee0463eb2 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:26.227302 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2c0b4-3a98-411d-9e36-837ee0463eb2 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:26.232067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88384ad8-1ccc-4175-9d2a-a6b19a954370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:26.232809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88384ad8-1ccc-4175-9d2a-a6b19a954370 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:26.233154 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88384ad8-1ccc-4175-9d2a-a6b19a954370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:26.233495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88384ad8-1ccc-4175-9d2a-a6b19a954370 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:26.234259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88384ad8-1ccc-4175-9d2a-a6b19a954370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:26.237523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88384ad8-1ccc-4175-9d2a-a6b19a954370 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:26.266100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1130/2254] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:31:26 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:31:26.270146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb45b5c-cf9d-44a0-ade7-0876d800af15 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:26.280858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb45b5c-cf9d-44a0-ade7-0876d800af15 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:26.350473 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e04c9-5d0a-4fca-ac4e-ea456ea3a6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:26.351193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e04c9-5d0a-4fca-ac4e-ea456ea3a6dc admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:26.351491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e04c9-5d0a-4fca-ac4e-ea456ea3a6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:26.351764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e04c9-5d0a-4fca-ac4e-ea456ea3a6dc admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:26.352461 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e04c9-5d0a-4fca-ac4e-ea456ea3a6dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:26.354423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e04c9-5d0a-4fca-ac4e-ea456ea3a6dc admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:26.378984 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1125/2255] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:31:26 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:31:26.380691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88384ad8-1ccc-4175-9d2a-a6b19a954370 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:26.392240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88384ad8-1ccc-4175-9d2a-a6b19a954370 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:26.397364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9d526-c943-4b93-a645-35b061c5c816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:26.398029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9d526-c943-4b93-a645-35b061c5c816 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:26.398315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9d526-c943-4b93-a645-35b061c5c816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:26.398611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9d526-c943-4b93-a645-35b061c5c816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:26.399340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9d526-c943-4b93-a645-35b061c5c816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:26.400984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9d526-c943-4b93-a645-35b061c5c816 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:26.416414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1131/2256] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:31:26.420555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e04c9-5d0a-4fca-ac4e-ea456ea3a6dc admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:26.432550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e04c9-5d0a-4fca-ac4e-ea456ea3a6dc admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:26.437836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b085c-60c5-4e0a-b1cc-537293d55298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:26.438507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b085c-60c5-4e0a-b1cc-537293d55298 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:26.438912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b085c-60c5-4e0a-b1cc-537293d55298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:26.439198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b085c-60c5-4e0a-b1cc-537293d55298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:26.439953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b085c-60c5-4e0a-b1cc-537293d55298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:26.441578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b085c-60c5-4e0a-b1cc-537293d55298 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:26.444125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:31:26.454972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1126/2257] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:31:26.458499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9d526-c943-4b93-a645-35b061c5c816 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:26.469767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9d526-c943-4b93-a645-35b061c5c816 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:26.474641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0062fa-fda0-49a5-b4fa-24375b8633af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:26.475514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0062fa-fda0-49a5-b4fa-24375b8633af admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:26.475791 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0062fa-fda0-49a5-b4fa-24375b8633af admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:26.476064 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0062fa-fda0-49a5-b4fa-24375b8633af admin admin] PATH_INFO: `/v3/projects/141bd2be69f243aa926f28b96eb6f0d5/users/7d53405e3f88439f81b67106ad98adc2/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:26.493264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0062fa-fda0-49a5-b4fa-24375b8633af admin admin] RBAC: Authorizing `identity:create_grant(project_id=141bd2be69f243aa926f28b96eb6f0d5, user_id=7d53405e3f88439f81b67106ad98adc2, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:26.496075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0062fa-fda0-49a5-b4fa-24375b8633af admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:26.513915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1132/2258] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:31:26 2022] PUT /identity/v3/projects/141bd2be69f243aa926f28b96eb6f0d5/users/7d53405e3f88439f81b67106ad98adc2/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:31:26.522680 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0c668-e873-4328-a267-a7027025e6f9 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:26.522974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0c668-e873-4328-a267-a7027025e6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:26.523252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0c668-e873-4328-a267-a7027025e6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:26.545401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c8a0c668-e873-4328-a267-a7027025e6f9 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:26.547036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c8a0c668-e873-4328-a267-a7027025e6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:26.609893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1127/2259] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:31:26.619347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0062fa-fda0-49a5-b4fa-24375b8633af admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:26.621050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ec0062fa-fda0-49a5-b4fa-24375b8633af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:26.684243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:31:26.689101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0062fa-fda0-49a5-b4fa-24375b8633af admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:26.695825 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1791257-ca6b-4b56-81e2-6757deda9503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:26.696709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791257-ca6b-4b56-81e2-6757deda9503 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:26.697197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791257-ca6b-4b56-81e2-6757deda9503 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:26.697555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1791257-ca6b-4b56-81e2-6757deda9503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:26.708867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1791257-ca6b-4b56-81e2-6757deda9503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:26.710353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1791257-ca6b-4b56-81e2-6757deda9503 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:26.720514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1133/2260] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:31:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:31:31.262565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f18a9c-42b1-4eb1-adae-086e077b15a0 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:31.263162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f18a9c-42b1-4eb1-adae-086e077b15a0 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:31.263432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f18a9c-42b1-4eb1-adae-086e077b15a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:31.296883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-67f18a9c-42b1-4eb1-adae-086e077b15a0 None None] MFA Rules not processed for user `7d53405e3f88439f81b67106ad98adc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:31.299149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-67f18a9c-42b1-4eb1-adae-086e077b15a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:31.406899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1128/2261] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:31:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:31:31.412174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1791257-ca6b-4b56-81e2-6757deda9503 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:31.421786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1791257-ca6b-4b56-81e2-6757deda9503 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:31.426576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c94bc0-c45b-425c-8f28-1f0cb7bc3680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:31.427484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c94bc0-c45b-425c-8f28-1f0cb7bc3680 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:31.427951 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c94bc0-c45b-425c-8f28-1f0cb7bc3680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:31.428385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c94bc0-c45b-425c-8f28-1f0cb7bc3680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:31.429492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c94bc0-c45b-425c-8f28-1f0cb7bc3680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:31.432842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c94bc0-c45b-425c-8f28-1f0cb7bc3680 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:31.473749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1134/2262] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:31:31 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:31:31.477738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f18a9c-42b1-4eb1-adae-086e077b15a0 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:31.490487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f18a9c-42b1-4eb1-adae-086e077b15a0 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:31.564341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f260180-7550-4a75-b820-27c36d718c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:31.565374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f260180-7550-4a75-b820-27c36d718c55 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:31.565670 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f260180-7550-4a75-b820-27c36d718c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:31.565971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f260180-7550-4a75-b820-27c36d718c55 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:31.566808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f260180-7550-4a75-b820-27c36d718c55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:31.569183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f260180-7550-4a75-b820-27c36d718c55 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:31.593700 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1129/2263] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:31:31 2022] POST /identity/v3/users => generated 296 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:31:31.597790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c94bc0-c45b-425c-8f28-1f0cb7bc3680 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:31.609141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c94bc0-c45b-425c-8f28-1f0cb7bc3680 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:31.616186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b848466-e80a-4a6c-a82b-b9a37db4b5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:31.617242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b848466-e80a-4a6c-a82b-b9a37db4b5bb admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:31.617608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b848466-e80a-4a6c-a82b-b9a37db4b5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:31.617987 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b848466-e80a-4a6c-a82b-b9a37db4b5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:31.618948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b848466-e80a-4a6c-a82b-b9a37db4b5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:31.620972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b848466-e80a-4a6c-a82b-b9a37db4b5bb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:31.633989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1135/2264] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:31:31.638184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f260180-7550-4a75-b820-27c36d718c55 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:31.649634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f260180-7550-4a75-b820-27c36d718c55 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:31.654379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb690c-f338-4f4f-97ba-9a5e8817461c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:31.655624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb690c-f338-4f4f-97ba-9a5e8817461c admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:31.655907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb690c-f338-4f4f-97ba-9a5e8817461c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:31.656212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb690c-f338-4f4f-97ba-9a5e8817461c admin admin] PATH_INFO: `/v3/projects/d0bc0a82cd0b4587b37873a83f39c3cf/users/138cb9d2d8884241aed9ebb55c9b814d/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:31.674878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb690c-f338-4f4f-97ba-9a5e8817461c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0bc0a82cd0b4587b37873a83f39c3cf, user_id=138cb9d2d8884241aed9ebb55c9b814d, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:31.677811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb690c-f338-4f4f-97ba-9a5e8817461c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:31.680488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:31:31.693038 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1130/2265] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:31:31 2022] PUT /identity/v3/projects/d0bc0a82cd0b4587b37873a83f39c3cf/users/138cb9d2d8884241aed9ebb55c9b814d/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:31:31.697769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b848466-e80a-4a6c-a82b-b9a37db4b5bb admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:31.709459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b848466-e80a-4a6c-a82b-b9a37db4b5bb admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:31.779932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e6d98-e5a8-4dfc-94a7-1da1647425d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:31.780625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e6d98-e5a8-4dfc-94a7-1da1647425d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:31.780953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e6d98-e5a8-4dfc-94a7-1da1647425d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:31.781260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e6d98-e5a8-4dfc-94a7-1da1647425d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:31.782758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e6d98-e5a8-4dfc-94a7-1da1647425d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:31.784489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e6d98-e5a8-4dfc-94a7-1da1647425d1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:31.798269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1136/2266] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:31:31.803349 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb690c-f338-4f4f-97ba-9a5e8817461c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:31.818215 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb690c-f338-4f4f-97ba-9a5e8817461c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:31.831685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565db9a-495b-493e-a167-c8b0006939c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:31.832881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565db9a-495b-493e-a167-c8b0006939c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:31.833289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565db9a-495b-493e-a167-c8b0006939c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:31.833690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565db9a-495b-493e-a167-c8b0006939c4 admin admin] PATH_INFO: `/v3/projects/d0bc0a82cd0b4587b37873a83f39c3cf/users/138cb9d2d8884241aed9ebb55c9b814d/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:31.836100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565db9a-495b-493e-a167-c8b0006939c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0bc0a82cd0b4587b37873a83f39c3cf, user_id=138cb9d2d8884241aed9ebb55c9b814d, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:31.839270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565db9a-495b-493e-a167-c8b0006939c4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:31.853493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1131/2267] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:31:31 2022] PUT /identity/v3/projects/d0bc0a82cd0b4587b37873a83f39c3cf/users/138cb9d2d8884241aed9ebb55c9b814d/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:31:36.876832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e6d98-e5a8-4dfc-94a7-1da1647425d1 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:36.878898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-950e6d98-e5a8-4dfc-94a7-1da1647425d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:36.913170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a91961-7f12-4769-8708-7ed8ac19ac78 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:36.913720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a91961-7f12-4769-8708-7ed8ac19ac78 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:36.914249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a91961-7f12-4769-8708-7ed8ac19ac78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:36.946164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c0a91961-7f12-4769-8708-7ed8ac19ac78 None None] MFA Rules not processed for user `138cb9d2d8884241aed9ebb55c9b814d`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:36.947528 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c0a91961-7f12-4769-8708-7ed8ac19ac78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:37.002784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e6d98-e5a8-4dfc-94a7-1da1647425d1 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:37.048186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1132/2268] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:31:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:31:37.049339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a91961-7f12-4769-8708-7ed8ac19ac78 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:37.051437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c0a91961-7f12-4769-8708-7ed8ac19ac78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:37.075042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e913784-c8bb-46c1-a335-d56b6a4cc32f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:37.075711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e913784-c8bb-46c1-a335-d56b6a4cc32f service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:37.075997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e913784-c8bb-46c1-a335-d56b6a4cc32f service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:37.076284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e913784-c8bb-46c1-a335-d56b6a4cc32f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:37.079402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2e913784-c8bb-46c1-a335-d56b6a4cc32f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:37.084784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:31:37.165983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a91961-7f12-4769-8708-7ed8ac19ac78 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:37.167913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e913784-c8bb-46c1-a335-d56b6a4cc32f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:37.169543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e913784-c8bb-46c1-a335-d56b6a4cc32f service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:37.173005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a05d6d-81fd-4c23-837f-e70198e791e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:37.173665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a05d6d-81fd-4c23-837f-e70198e791e0 service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:37.173946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a05d6d-81fd-4c23-837f-e70198e791e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:37.174220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a05d6d-81fd-4c23-837f-e70198e791e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:37.183817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1137/2269] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:31:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:31:37.249098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a05d6d-81fd-4c23-837f-e70198e791e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:37.251237 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a05d6d-81fd-4c23-837f-e70198e791e0 service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:37.262893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1133/2270] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:31:37.414978 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e913784-c8bb-46c1-a335-d56b6a4cc32f service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:37.416697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2e913784-c8bb-46c1-a335-d56b6a4cc32f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:37.554328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e913784-c8bb-46c1-a335-d56b6a4cc32f service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:37.560544 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf2b1d-490e-4fd7-9788-a7a5667bb35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:37.561568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf2b1d-490e-4fd7-9788-a7a5667bb35d service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:37.562057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf2b1d-490e-4fd7-9788-a7a5667bb35d service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:37.562666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cf2b1d-490e-4fd7-9788-a7a5667bb35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:37.567590 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-26cf2b1d-490e-4fd7-9788-a7a5667bb35d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:37.599659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf2b1d-490e-4fd7-9788-a7a5667bb35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:37.601451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cf2b1d-490e-4fd7-9788-a7a5667bb35d service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:37.629807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1138/2271] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:31:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:31:47.257001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:INSERT=2,keystone:SELECT=97 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:31:47.613960 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=57 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:31:49.175179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a05d6d-81fd-4c23-837f-e70198e791e0 service nova] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:49.185194 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a05d6d-81fd-4c23-837f-e70198e791e0 service nova] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:49.189714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdd83e-aa7c-4bb4-8626-09636eb83cfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:49.190480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdd83e-aa7c-4bb4-8626-09636eb83cfc service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:49.191087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdd83e-aa7c-4bb4-8626-09636eb83cfc service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:49.191241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdd83e-aa7c-4bb4-8626-09636eb83cfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:49.193413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3bcdd83e-aa7c-4bb4-8626-09636eb83cfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:49.282898 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdd83e-aa7c-4bb4-8626-09636eb83cfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:49.284502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdd83e-aa7c-4bb4-8626-09636eb83cfc service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:49.296282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1134/2272] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:31:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:31:51.496576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf2b1d-490e-4fd7-9788-a7a5667bb35d service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:51.505007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cf2b1d-490e-4fd7-9788-a7a5667bb35d service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:51.508556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327af9e0-6d79-4e7f-8574-8884f8f2a395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:51.509181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327af9e0-6d79-4e7f-8574-8884f8f2a395 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:51.509459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327af9e0-6d79-4e7f-8574-8884f8f2a395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:51.509729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327af9e0-6d79-4e7f-8574-8884f8f2a395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:51.511612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-327af9e0-6d79-4e7f-8574-8884f8f2a395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:51.577663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327af9e0-6d79-4e7f-8574-8884f8f2a395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:51.579848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327af9e0-6d79-4e7f-8574-8884f8f2a395 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:51.589222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1139/2273] 10.222.0.93 () {60 vars in 1396 bytes} [Fri Jun 3 15:31:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:31:52.018998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdd83e-aa7c-4bb4-8626-09636eb83cfc service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:52.030280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdd83e-aa7c-4bb4-8626-09636eb83cfc service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:52.033864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d576c-718c-4936-bff5-47cd016b7e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:52.034511 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d576c-718c-4936-bff5-47cd016b7e21 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:52.034896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d576c-718c-4936-bff5-47cd016b7e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:52.035176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d576c-718c-4936-bff5-47cd016b7e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:52.037068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-899d576c-718c-4936-bff5-47cd016b7e21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:52.052245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d576c-718c-4936-bff5-47cd016b7e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:52.053815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d576c-718c-4936-bff5-47cd016b7e21 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:52.064362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1135/2274] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:31:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:31:54.066204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327af9e0-6d79-4e7f-8574-8884f8f2a395 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:54.092051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327af9e0-6d79-4e7f-8574-8884f8f2a395 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:54.111077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525203a4-5b18-4405-9ec3-0960edb54cca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:54.112049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525203a4-5b18-4405-9ec3-0960edb54cca service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:54.112501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525203a4-5b18-4405-9ec3-0960edb54cca service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:54.112934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525203a4-5b18-4405-9ec3-0960edb54cca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:54.123710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-525203a4-5b18-4405-9ec3-0960edb54cca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:54.142710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525203a4-5b18-4405-9ec3-0960edb54cca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:54.149101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525203a4-5b18-4405-9ec3-0960edb54cca service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:54.168730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1140/2275] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:31:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:31:55.349890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a895438-e902-4981-8861-de0ba19189e3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:55.350448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a895438-e902-4981-8861-de0ba19189e3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:55.350576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a895438-e902-4981-8861-de0ba19189e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:55.379505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-5a895438-e902-4981-8861-de0ba19189e3 None None] MFA Rules not processed for user `193a35a91a47410f8b12cb2e84b33e1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:55.381133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5a895438-e902-4981-8861-de0ba19189e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:55.395234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1136/2276] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:31:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:31:55.721042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525203a4-5b18-4405-9ec3-0960edb54cca service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:55.730276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525203a4-5b18-4405-9ec3-0960edb54cca service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:55.734161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50c6dd-e99c-40f3-9e29-0e0ff2a9afc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:55.734913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50c6dd-e99c-40f3-9e29-0e0ff2a9afc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:55.735198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50c6dd-e99c-40f3-9e29-0e0ff2a9afc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:55.735474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50c6dd-e99c-40f3-9e29-0e0ff2a9afc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:55.737252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3b50c6dd-e99c-40f3-9e29-0e0ff2a9afc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:55.747613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50c6dd-e99c-40f3-9e29-0e0ff2a9afc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:55.750238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50c6dd-e99c-40f3-9e29-0e0ff2a9afc3 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:55.759761 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1141/2277] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:31:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:31:56.305642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a895438-e902-4981-8861-de0ba19189e3 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:56.307354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5a895438-e902-4981-8861-de0ba19189e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:56.387640 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a895438-e902-4981-8861-de0ba19189e3 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:56.392991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eff61e-9845-4fb7-a37c-ac35f07f4444 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:56.393614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff61e-9845-4fb7-a37c-ac35f07f4444 service placement] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:56.393893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff61e-9845-4fb7-a37c-ac35f07f4444 service placement] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:56.394168 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff61e-9845-4fb7-a37c-ac35f07f4444 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:56.395940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-08eff61e-9845-4fb7-a37c-ac35f07f4444 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:56.408254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff61e-9845-4fb7-a37c-ac35f07f4444 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:56.409938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff61e-9845-4fb7-a37c-ac35f07f4444 service placement] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:56.420116 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1137/2278] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:31:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:31:56.621444 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7fbef-f6ae-43be-b020-99dd6f79468a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:56.621748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7fbef-f6ae-43be-b020-99dd6f79468a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:56.622047 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a7fbef-f6ae-43be-b020-99dd6f79468a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:56.647062 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a1a7fbef-f6ae-43be-b020-99dd6f79468a None None] MFA Rules not processed for user `7d53405e3f88439f81b67106ad98adc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:56.648807 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a1a7fbef-f6ae-43be-b020-99dd6f79468a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:56.661441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1142/2279] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:31:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:31:57.824671 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60f69-032f-41e6-8ff6-7f37e470f2a6 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:57.825323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60f69-032f-41e6-8ff6-7f37e470f2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:57.825323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60f69-032f-41e6-8ff6-7f37e470f2a6 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:57.828150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1138/2280] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:31:57.845251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e1e32-51aa-496d-aecc-63894add73f2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:57.845866 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e1e32-51aa-496d-aecc-63894add73f2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:57.845965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e1e32-51aa-496d-aecc-63894add73f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:57.878595 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d02e1e32-51aa-496d-aecc-63894add73f2 None None] MFA Rules not processed for user `e3f887d5bbcd424884537b95affd96f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:57.880430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d02e1e32-51aa-496d-aecc-63894add73f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:57.891482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1143/2281] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:31:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:31:57.905221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a60f69-032f-41e6-8ff6-7f37e470f2a6 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:57.921701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a60f69-032f-41e6-8ff6-7f37e470f2a6 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:57.932868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d1802-2bb8-498a-800c-35a0daa0da37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:57.933719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d1802-2bb8-498a-800c-35a0daa0da37 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:57.934285 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d1802-2bb8-498a-800c-35a0daa0da37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:57.934752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d1802-2bb8-498a-800c-35a0daa0da37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:57.961414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d1802-2bb8-498a-800c-35a0daa0da37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:57.962991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d1802-2bb8-498a-800c-35a0daa0da37 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:57.990836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1139/2282] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:31:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:31:58.143401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86751e75-ab26-46d0-b053-1377da5860c4 None None] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:58.143773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86751e75-ab26-46d0-b053-1377da5860c4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:58.144207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86751e75-ab26-46d0-b053-1377da5860c4 None None] PATH_INFO: `/` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:58.145985 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1144/2283] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:31:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:31:58.155299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cd63e-9309-49f9-8752-b60920cac145 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:58.156693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cd63e-9309-49f9-8752-b60920cac145 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:58.157128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8cd63e-9309-49f9-8752-b60920cac145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:58.181611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2f8cd63e-9309-49f9-8752-b60920cac145 None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:31:58.183030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2f8cd63e-9309-49f9-8752-b60920cac145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:31:58.196393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1140/2284] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:31:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:31:58.208734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86751e75-ab26-46d0-b053-1377da5860c4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:31:58.223342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86751e75-ab26-46d0-b053-1377da5860c4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:31:58.228745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17aa459-4fae-4308-b9a4-14deb4f32bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:31:58.229720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17aa459-4fae-4308-b9a4-14deb4f32bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:58.230242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17aa459-4fae-4308-b9a4-14deb4f32bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:58.230763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17aa459-4fae-4308-b9a4-14deb4f32bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:31:58.246562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17aa459-4fae-4308-b9a4-14deb4f32bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:31:58.248314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17aa459-4fae-4308-b9a4-14deb4f32bc6 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:31:58.271992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1145/2285] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:31:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:31:59.977742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b851-8646-47b2-a225-3b66d28c1f49 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:31:59.978278 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b851-8646-47b2-a225-3b66d28c1f49 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:31:59.978405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7b851-8646-47b2-a225-3b66d28c1f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:00.005225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-29a7b851-8646-47b2-a225-3b66d28c1f49 None None] MFA Rules not processed for user `8632496bdc1b4d06a74caf483c8b0a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:00.006620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-29a7b851-8646-47b2-a225-3b66d28c1f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:00.014742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1141/2286] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:31:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:05.047122 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17aa459-4fae-4308-b9a4-14deb4f32bc6 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:05.049364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f17aa459-4fae-4308-b9a4-14deb4f32bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:05.065234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17aa459-4fae-4308-b9a4-14deb4f32bc6 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:05.071783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d19c1-5ea9-414a-9da4-330ac0cb7fe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:05.072713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d19c1-5ea9-414a-9da4-330ac0cb7fe4 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:05.073165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d19c1-5ea9-414a-9da4-330ac0cb7fe4 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:05.073611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d19c1-5ea9-414a-9da4-330ac0cb7fe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:05.075664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a28d19c1-5ea9-414a-9da4-330ac0cb7fe4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:05.176520 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d19c1-5ea9-414a-9da4-330ac0cb7fe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:05.179471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d19c1-5ea9-414a-9da4-330ac0cb7fe4 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:05.203015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1146/2287] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:09.998840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=81 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:14.201208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7b851-8646-47b2-a225-3b66d28c1f49 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:14.216994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7b851-8646-47b2-a225-3b66d28c1f49 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:14.224019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5e29e-3194-4cc7-aab6-25db2a6dbaac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:14.225015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e29e-3194-4cc7-aab6-25db2a6dbaac service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:14.225516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e29e-3194-4cc7-aab6-25db2a6dbaac service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:14.226162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e29e-3194-4cc7-aab6-25db2a6dbaac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:14.228604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-faf5e29e-3194-4cc7-aab6-25db2a6dbaac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:14.315799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5e29e-3194-4cc7-aab6-25db2a6dbaac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:14.317625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5e29e-3194-4cc7-aab6-25db2a6dbaac service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:14.328728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1142/2288] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:14.512509 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d19c1-5ea9-414a-9da4-330ac0cb7fe4 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:14.524210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d19c1-5ea9-414a-9da4-330ac0cb7fe4 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:14.531100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e67f1c-2209-4905-a8be-7ce5f9fa72d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:14.531748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e67f1c-2209-4905-a8be-7ce5f9fa72d1 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:14.532032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e67f1c-2209-4905-a8be-7ce5f9fa72d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:14.532308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e67f1c-2209-4905-a8be-7ce5f9fa72d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:14.534074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-49e67f1c-2209-4905-a8be-7ce5f9fa72d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:14.547497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e67f1c-2209-4905-a8be-7ce5f9fa72d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:14.549226 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e67f1c-2209-4905-a8be-7ce5f9fa72d1 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:14.562363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1147/2289] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:23.554297 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5e29e-3194-4cc7-aab6-25db2a6dbaac service nova] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:23.566877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5e29e-3194-4cc7-aab6-25db2a6dbaac service nova] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:23.572967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247fd0f-ea22-41f1-91f6-1d6b156d83de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:23.573618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247fd0f-ea22-41f1-91f6-1d6b156d83de service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:23.573897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247fd0f-ea22-41f1-91f6-1d6b156d83de service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:23.574176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247fd0f-ea22-41f1-91f6-1d6b156d83de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:23.576061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3247fd0f-ea22-41f1-91f6-1d6b156d83de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:23.589519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247fd0f-ea22-41f1-91f6-1d6b156d83de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:23.591183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247fd0f-ea22-41f1-91f6-1d6b156d83de service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:23.603057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1143/2290] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:32:24.191256 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e67f1c-2209-4905-a8be-7ce5f9fa72d1 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:24.195846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=87 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:24.203047 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e67f1c-2209-4905-a8be-7ce5f9fa72d1 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:24.208790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f502d-e9b2-4b49-a631-cc3ec6a2ed76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:24.209563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f502d-e9b2-4b49-a631-cc3ec6a2ed76 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:24.209977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f502d-e9b2-4b49-a631-cc3ec6a2ed76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:24.210491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f502d-e9b2-4b49-a631-cc3ec6a2ed76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:24.227763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f502d-e9b2-4b49-a631-cc3ec6a2ed76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:24.229947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f502d-e9b2-4b49-a631-cc3ec6a2ed76 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:24.253438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1148/2291] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:24.286372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247fd0f-ea22-41f1-91f6-1d6b156d83de service nova] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:24.299436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247fd0f-ea22-41f1-91f6-1d6b156d83de service nova] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:24.308426 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5abd4f-9bc6-417d-8b16-4a24fc66d90c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:24.309089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5abd4f-9bc6-417d-8b16-4a24fc66d90c service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:24.309370 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5abd4f-9bc6-417d-8b16-4a24fc66d90c service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:24.309645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5abd4f-9bc6-417d-8b16-4a24fc66d90c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:24.317440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3e5abd4f-9bc6-417d-8b16-4a24fc66d90c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:24.443326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5abd4f-9bc6-417d-8b16-4a24fc66d90c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:24.444787 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5abd4f-9bc6-417d-8b16-4a24fc66d90c service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:24.466217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1144/2292] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:29.587178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f502d-e9b2-4b49-a631-cc3ec6a2ed76 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:29.589582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6d2f502d-e9b2-4b49-a631-cc3ec6a2ed76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:29.607030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f502d-e9b2-4b49-a631-cc3ec6a2ed76 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:29.611803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203c0b1a-345a-4c2f-a04a-a9e4dc164cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:29.612833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c0b1a-345a-4c2f-a04a-a9e4dc164cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:29.613236 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c0b1a-345a-4c2f-a04a-a9e4dc164cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:29.613511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203c0b1a-345a-4c2f-a04a-a9e4dc164cd4 admin admin] PATH_INFO: `/v3/users/127a6938bc5c4416a9a502d9064344cd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:29.615245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c0b1a-345a-4c2f-a04a-a9e4dc164cd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127a6938bc5c4416a9a502d9064344cd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:29.617207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203c0b1a-345a-4c2f-a04a-a9e4dc164cd4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:29.680732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-203c0b1a-345a-4c2f-a04a-a9e4dc164cd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127a6938bc5c4416a9a502d9064344cd'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:29.692770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-203c0b1a-345a-4c2f-a04a-a9e4dc164cd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127a6938bc5c4416a9a502d9064344cd'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:29.693372 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-203c0b1a-345a-4c2f-a04a-a9e4dc164cd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127a6938bc5c4416a9a502d9064344cd'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:29.701078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-203c0b1a-345a-4c2f-a04a-a9e4dc164cd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127a6938bc5c4416a9a502d9064344cd'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:29.701713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-203c0b1a-345a-4c2f-a04a-a9e4dc164cd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127a6938bc5c4416a9a502d9064344cd'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:29.714677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1149/2293] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:32:29 2022] DELETE /identity/v3/users/127a6938bc5c4416a9a502d9064344cd => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:29.896321 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5abd4f-9bc6-417d-8b16-4a24fc66d90c service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:29.908187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5abd4f-9bc6-417d-8b16-4a24fc66d90c service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:29.970886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:29.971753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:29.972030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:29.972299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] PATH_INFO: `/v3/projects/a402ec1153624838be07cda58279f2a4` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:29.973484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a402ec1153624838be07cda58279f2a4)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:29.975440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:29.997751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:30.020327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a402ec1153624838be07cda58279f2a4', 'request_id': 'req-8b4f5852-0808-49c9-9d0c-61d439ef4391'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:30.025741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a402ec1153624838be07cda58279f2a4', 'request_id': 'req-8b4f5852-0808-49c9-9d0c-61d439ef4391'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:30.033733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a402ec1153624838be07cda58279f2a4', 'request_id': 'req-8b4f5852-0808-49c9-9d0c-61d439ef4391'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:30.044577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] The token cache is being invalidate because project a402ec1153624838be07cda58279f2a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:32:30.044866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:30.045505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8b4f5852-0808-49c9-9d0c-61d439ef4391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:30.047102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1145/2294] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:32:29 2022] DELETE /identity/v3/projects/a402ec1153624838be07cda58279f2a4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:30.053205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729da9f-94c9-42e0-b508-57bb3de81a9b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.053438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729da9f-94c9-42e0-b508-57bb3de81a9b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.053720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729da9f-94c9-42e0-b508-57bb3de81a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.071484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-3729da9f-94c9-42e0-b508-57bb3de81a9b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:30.072682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3729da9f-94c9-42e0-b508-57bb3de81a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:30.135567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1150/2295] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:30.144314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b77fa-e60b-4b9e-a573-96ad600e4fb6 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.144603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b77fa-e60b-4b9e-a573-96ad600e4fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.144882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b77fa-e60b-4b9e-a573-96ad600e4fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.164160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-312b77fa-e60b-4b9e-a573-96ad600e4fb6 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:30.165405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-312b77fa-e60b-4b9e-a573-96ad600e4fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:30.177771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1146/2296] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:30.184040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729da9f-94c9-42e0-b508-57bb3de81a9b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:30.196643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729da9f-94c9-42e0-b508-57bb3de81a9b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:30.201907 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde75142-0792-4a3a-84bb-38fc58581ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:30.202617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde75142-0792-4a3a-84bb-38fc58581ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.202911 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde75142-0792-4a3a-84bb-38fc58581ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.203189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde75142-0792-4a3a-84bb-38fc58581ed4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.203918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde75142-0792-4a3a-84bb-38fc58581ed4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:32:30.204223 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde75142-0792-4a3a-84bb-38fc58581ed4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:30.205866 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde75142-0792-4a3a-84bb-38fc58581ed4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:30.217763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1151/2297] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:32:30.222037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b77fa-e60b-4b9e-a573-96ad600e4fb6 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:30.231457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b77fa-e60b-4b9e-a573-96ad600e4fb6 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:30.236147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c251ee-1942-4971-bdf4-097cbf96a4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:30.236790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c251ee-1942-4971-bdf4-097cbf96a4fe admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.237071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c251ee-1942-4971-bdf4-097cbf96a4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.237342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c251ee-1942-4971-bdf4-097cbf96a4fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.238023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c251ee-1942-4971-bdf4-097cbf96a4fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:30.239947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c251ee-1942-4971-bdf4-097cbf96a4fe admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:30.266558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1147/2298] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:32:30 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:32:30.270653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde75142-0792-4a3a-84bb-38fc58581ed4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:30.280623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde75142-0792-4a3a-84bb-38fc58581ed4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:30.324976 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:30.339140 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497296df-98a4-48a5-b31d-4a869bd48b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:30.339812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497296df-98a4-48a5-b31d-4a869bd48b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.340095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497296df-98a4-48a5-b31d-4a869bd48b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.340375 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497296df-98a4-48a5-b31d-4a869bd48b32 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.341057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497296df-98a4-48a5-b31d-4a869bd48b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:30.343039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497296df-98a4-48a5-b31d-4a869bd48b32 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:30.362993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1152/2299] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:32:30 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:32:30.367036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c251ee-1942-4971-bdf4-097cbf96a4fe admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:30.377916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c251ee-1942-4971-bdf4-097cbf96a4fe admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:30.382548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eb3bd-3cd9-44f0-aa0d-2f18f60ecfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:30.383200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eb3bd-3cd9-44f0-aa0d-2f18f60ecfc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.383484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eb3bd-3cd9-44f0-aa0d-2f18f60ecfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.383759 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eb3bd-3cd9-44f0-aa0d-2f18f60ecfc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.384521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96eb3bd-3cd9-44f0-aa0d-2f18f60ecfc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:30.386122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96eb3bd-3cd9-44f0-aa0d-2f18f60ecfc4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:30.397176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1148/2300] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:30.400874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497296df-98a4-48a5-b31d-4a869bd48b32 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:30.410860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497296df-98a4-48a5-b31d-4a869bd48b32 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:30.415389 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74982d49-a9db-4748-93bc-5c47901f7a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:30.416034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74982d49-a9db-4748-93bc-5c47901f7a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.416577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74982d49-a9db-4748-93bc-5c47901f7a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.416863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74982d49-a9db-4748-93bc-5c47901f7a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.417583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74982d49-a9db-4748-93bc-5c47901f7a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:30.419257 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74982d49-a9db-4748-93bc-5c47901f7a17 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:30.429711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1153/2301] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:30.433970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eb3bd-3cd9-44f0-aa0d-2f18f60ecfc4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:30.444226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eb3bd-3cd9-44f0-aa0d-2f18f60ecfc4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:30.448941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9631ddf-16bc-41a5-bd39-5e751188db9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:30.449821 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631ddf-16bc-41a5-bd39-5e751188db9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.450095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631ddf-16bc-41a5-bd39-5e751188db9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.450365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9631ddf-16bc-41a5-bd39-5e751188db9b admin admin] PATH_INFO: `/v3/projects/87500b5c68f54daeb650300bc8d485e8/users/1326bf3c8535486aa857fa98d6ea52d7/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.465113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9631ddf-16bc-41a5-bd39-5e751188db9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=87500b5c68f54daeb650300bc8d485e8, user_id=1326bf3c8535486aa857fa98d6ea52d7, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:30.467946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9631ddf-16bc-41a5-bd39-5e751188db9b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:30.480847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1149/2302] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:32:30 2022] PUT /identity/v3/projects/87500b5c68f54daeb650300bc8d485e8/users/1326bf3c8535486aa857fa98d6ea52d7/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:32:30.491534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed9649-1ff6-4b6d-bf0d-26ee4b54e582 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.491905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed9649-1ff6-4b6d-bf0d-26ee4b54e582 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.492255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed9649-1ff6-4b6d-bf0d-26ee4b54e582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.520456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c5ed9649-1ff6-4b6d-bf0d-26ee4b54e582 None None] MFA Rules not processed for user `1326bf3c8535486aa857fa98d6ea52d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:30.521656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c5ed9649-1ff6-4b6d-bf0d-26ee4b54e582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:30.588422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1154/2303] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:30.592887 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9631ddf-16bc-41a5-bd39-5e751188db9b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:30.655348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9631ddf-16bc-41a5-bd39-5e751188db9b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:30.659974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f85e7-cc6b-419e-99ce-b43c623a1d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:30.660627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f85e7-cc6b-419e-99ce-b43c623a1d41 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.660908 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f85e7-cc6b-419e-99ce-b43c623a1d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.661181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f85e7-cc6b-419e-99ce-b43c623a1d41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.661864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f85e7-cc6b-419e-99ce-b43c623a1d41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:30.663809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f85e7-cc6b-419e-99ce-b43c623a1d41 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:30.694367 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1150/2304] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:32:30 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:32:30.695428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed9649-1ff6-4b6d-bf0d-26ee4b54e582 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:30.707042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed9649-1ff6-4b6d-bf0d-26ee4b54e582 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:30.765474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60158320-08a4-44c2-a033-454d171cd7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:30.766189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60158320-08a4-44c2-a033-454d171cd7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.766472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60158320-08a4-44c2-a033-454d171cd7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.766802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60158320-08a4-44c2-a033-454d171cd7b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.767491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60158320-08a4-44c2-a033-454d171cd7b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:30.769417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60158320-08a4-44c2-a033-454d171cd7b7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:30.790156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1155/2305] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:32:30 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:32:30.794076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f85e7-cc6b-419e-99ce-b43c623a1d41 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:30.805744 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f85e7-cc6b-419e-99ce-b43c623a1d41 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:30.811963 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7e75c-dc4b-41b2-9274-e9207ee264f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:30.812729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e75c-dc4b-41b2-9274-e9207ee264f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.813019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e75c-dc4b-41b2-9274-e9207ee264f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.813331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e75c-dc4b-41b2-9274-e9207ee264f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.814096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7e75c-dc4b-41b2-9274-e9207ee264f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:30.815779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d7e75c-dc4b-41b2-9274-e9207ee264f8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:30.826796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1151/2306] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:30.830763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60158320-08a4-44c2-a033-454d171cd7b7 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:30.841129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60158320-08a4-44c2-a033-454d171cd7b7 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:30.845873 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d20b8-1c1e-4376-aaaf-20b527073f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:30.846550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d20b8-1c1e-4376-aaaf-20b527073f25 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.846836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d20b8-1c1e-4376-aaaf-20b527073f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.847107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d20b8-1c1e-4376-aaaf-20b527073f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.847836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d20b8-1c1e-4376-aaaf-20b527073f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:30.849399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d20b8-1c1e-4376-aaaf-20b527073f25 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:30.860220 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1156/2307] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:30.864420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7e75c-dc4b-41b2-9274-e9207ee264f8 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:30.875095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d7e75c-dc4b-41b2-9274-e9207ee264f8 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:30.879709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb209ee-0752-4b20-a0ea-df9b92c683e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:30.880608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb209ee-0752-4b20-a0ea-df9b92c683e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.880890 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb209ee-0752-4b20-a0ea-df9b92c683e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.881168 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb209ee-0752-4b20-a0ea-df9b92c683e5 admin admin] PATH_INFO: `/v3/projects/78101a62bbb1443c8b36470a57ce9d10/users/cc2a6ce885e24fac89c74f07755683a7/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.896687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb209ee-0752-4b20-a0ea-df9b92c683e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78101a62bbb1443c8b36470a57ce9d10, user_id=cc2a6ce885e24fac89c74f07755683a7, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:30.899402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb209ee-0752-4b20-a0ea-df9b92c683e5 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:30.902423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:30.914833 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1152/2308] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:32:30 2022] PUT /identity/v3/projects/78101a62bbb1443c8b36470a57ce9d10/users/cc2a6ce885e24fac89c74f07755683a7/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:30.925686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d4f94-6fab-46f6-ba74-5f187476bc88 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:30.925975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d4f94-6fab-46f6-ba74-5f187476bc88 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:30.926251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d4f94-6fab-46f6-ba74-5f187476bc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:30.951790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-aa9d4f94-6fab-46f6-ba74-5f187476bc88 None None] MFA Rules not processed for user `cc2a6ce885e24fac89c74f07755683a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:30.952994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aa9d4f94-6fab-46f6-ba74-5f187476bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:30.976500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:31.017161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1157/2309] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:31.021702 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb209ee-0752-4b20-a0ea-df9b92c683e5 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:31.082038 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb209ee-0752-4b20-a0ea-df9b92c683e5 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:31.086685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fd156-7d67-4cc8-b7e4-465036ea6b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:31.087331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd156-7d67-4cc8-b7e4-465036ea6b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:31.087613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd156-7d67-4cc8-b7e4-465036ea6b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:31.087916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd156-7d67-4cc8-b7e4-465036ea6b2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:31.088609 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529fd156-7d67-4cc8-b7e4-465036ea6b2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:31.090565 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529fd156-7d67-4cc8-b7e4-465036ea6b2a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:31.117870 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1153/2310] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:32:31 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:32:31.121981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d4f94-6fab-46f6-ba74-5f187476bc88 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:31.132577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9d4f94-6fab-46f6-ba74-5f187476bc88 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:31.188740 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42d158-4cd6-4a29-b349-86947b293b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:31.189405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d158-4cd6-4a29-b349-86947b293b03 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:31.189683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d158-4cd6-4a29-b349-86947b293b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:31.189958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d158-4cd6-4a29-b349-86947b293b03 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:31.190669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42d158-4cd6-4a29-b349-86947b293b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:31.192615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42d158-4cd6-4a29-b349-86947b293b03 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:31.213128 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1158/2311] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:32:31 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:32:31.217882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fd156-7d67-4cc8-b7e4-465036ea6b2a admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:31.227959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fd156-7d67-4cc8-b7e4-465036ea6b2a admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:31.233762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb34d5-88f7-441a-b968-0e79870869e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:31.234413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb34d5-88f7-441a-b968-0e79870869e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:31.234729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb34d5-88f7-441a-b968-0e79870869e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:31.235003 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb34d5-88f7-441a-b968-0e79870869e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:31.235724 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb34d5-88f7-441a-b968-0e79870869e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:31.237311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cb34d5-88f7-441a-b968-0e79870869e4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:31.247909 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1154/2312] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:31.252234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42d158-4cd6-4a29-b349-86947b293b03 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:31.262022 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42d158-4cd6-4a29-b349-86947b293b03 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:31.267518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dd177-19ff-4d01-80a5-28f2b31561fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:31.268382 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dd177-19ff-4d01-80a5-28f2b31561fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:31.268695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dd177-19ff-4d01-80a5-28f2b31561fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:31.268968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dd177-19ff-4d01-80a5-28f2b31561fb admin admin] PATH_INFO: `/v3/projects/3218a96a94084f1888d62d5c9eaf2815/users/c6a7fba2c87d4ff2b49e01b87f8dfbd1/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:31.283883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709dd177-19ff-4d01-80a5-28f2b31561fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3218a96a94084f1888d62d5c9eaf2815, user_id=c6a7fba2c87d4ff2b49e01b87f8dfbd1, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:31.286580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709dd177-19ff-4d01-80a5-28f2b31561fb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:31.299546 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1159/2313] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:32:31 2022] PUT /identity/v3/projects/3218a96a94084f1888d62d5c9eaf2815/users/c6a7fba2c87d4ff2b49e01b87f8dfbd1/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:31.303513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb34d5-88f7-441a-b968-0e79870869e4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:31.313484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb34d5-88f7-441a-b968-0e79870869e4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:31.370985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e3aaf-5869-46e0-86e0-afb9f075b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:31.371570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e3aaf-5869-46e0-86e0-afb9f075b871 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:31.371857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e3aaf-5869-46e0-86e0-afb9f075b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:31.372133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e3aaf-5869-46e0-86e0-afb9f075b871 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:31.372898 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e3aaf-5869-46e0-86e0-afb9f075b871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:31.374511 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e3aaf-5869-46e0-86e0-afb9f075b871 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:31.385052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1155/2314] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:31.390101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dd177-19ff-4d01-80a5-28f2b31561fb admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:31.400264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dd177-19ff-4d01-80a5-28f2b31561fb admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:31.405674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a5f74-eea2-4832-b5b5-6965485bfee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:31.406562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a5f74-eea2-4832-b5b5-6965485bfee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:31.406852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a5f74-eea2-4832-b5b5-6965485bfee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:31.407126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a5f74-eea2-4832-b5b5-6965485bfee2 admin admin] PATH_INFO: `/v3/projects/3218a96a94084f1888d62d5c9eaf2815/users/c6a7fba2c87d4ff2b49e01b87f8dfbd1/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:31.409081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a5f74-eea2-4832-b5b5-6965485bfee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3218a96a94084f1888d62d5c9eaf2815, user_id=c6a7fba2c87d4ff2b49e01b87f8dfbd1, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:31.411793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a5f74-eea2-4832-b5b5-6965485bfee2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:31.421934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1160/2315] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:32:31 2022] PUT /identity/v3/projects/3218a96a94084f1888d62d5c9eaf2815/users/c6a7fba2c87d4ff2b49e01b87f8dfbd1/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:31.433059 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1e00-a005-4f74-9a0d-70a1583d559c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:31.433347 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1e00-a005-4f74-9a0d-70a1583d559c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:31.433625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec1e00-a005-4f74-9a0d-70a1583d559c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:31.461903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-17ec1e00-a005-4f74-9a0d-70a1583d559c None None] MFA Rules not processed for user `c6a7fba2c87d4ff2b49e01b87f8dfbd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:31.463165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-17ec1e00-a005-4f74-9a0d-70a1583d559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:31.515340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:31.534493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1156/2316] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:31.545553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a5f74-eea2-4832-b5b5-6965485bfee2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:31.547249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b32a5f74-eea2-4832-b5b5-6965485bfee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:31.607685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a5f74-eea2-4832-b5b5-6965485bfee2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:31.611390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b284b-7fd6-45a6-9a35-a86cf0063d46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:31.612014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b284b-7fd6-45a6-9a35-a86cf0063d46 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:31.612293 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b284b-7fd6-45a6-9a35-a86cf0063d46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:31.612564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230b284b-7fd6-45a6-9a35-a86cf0063d46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:31.622626 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b284b-7fd6-45a6-9a35-a86cf0063d46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:31.623995 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230b284b-7fd6-45a6-9a35-a86cf0063d46 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:31.633489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1161/2317] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:31.674139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec1e00-a005-4f74-9a0d-70a1583d559c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:31.675583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-17ec1e00-a005-4f74-9a0d-70a1583d559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:31.687968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec1e00-a005-4f74-9a0d-70a1583d559c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:31.691844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbb922-bbd6-4e61-a098-cbd9842271bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:31.692537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbb922-bbd6-4e61-a098-cbd9842271bb service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:31.692823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbb922-bbd6-4e61-a098-cbd9842271bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:31.693099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbb922-bbd6-4e61-a098-cbd9842271bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:31.749068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbb922-bbd6-4e61-a098-cbd9842271bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:31.750525 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbb922-bbd6-4e61-a098-cbd9842271bb service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:31.759473 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1157/2318] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:34.634287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b284b-7fd6-45a6-9a35-a86cf0063d46 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:34.665885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:34.715930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230b284b-7fd6-45a6-9a35-a86cf0063d46 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:34.720783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a97dac-7315-4084-9d6d-f785d1155486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:34.721647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a97dac-7315-4084-9d6d-f785d1155486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:34.721925 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a97dac-7315-4084-9d6d-f785d1155486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:34.722198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a97dac-7315-4084-9d6d-f785d1155486 admin admin] PATH_INFO: `/v3/users/1326bf3c8535486aa857fa98d6ea52d7` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:34.723427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a97dac-7315-4084-9d6d-f785d1155486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1326bf3c8535486aa857fa98d6ea52d7)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:34.725371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a97dac-7315-4084-9d6d-f785d1155486 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:34.776086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-89a97dac-7315-4084-9d6d-f785d1155486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1326bf3c8535486aa857fa98d6ea52d7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:34.788833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-89a97dac-7315-4084-9d6d-f785d1155486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1326bf3c8535486aa857fa98d6ea52d7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:34.789467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-89a97dac-7315-4084-9d6d-f785d1155486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1326bf3c8535486aa857fa98d6ea52d7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:34.796706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-89a97dac-7315-4084-9d6d-f785d1155486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1326bf3c8535486aa857fa98d6ea52d7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:34.797363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-89a97dac-7315-4084-9d6d-f785d1155486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1326bf3c8535486aa857fa98d6ea52d7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:34.811500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1162/2319] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:32:34 2022] DELETE /identity/v3/users/1326bf3c8535486aa857fa98d6ea52d7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:34.816622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73125c-09da-4791-a085-e5b5548c77ea None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:34.817115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73125c-09da-4791-a085-e5b5548c77ea None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:34.817331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73125c-09da-4791-a085-e5b5548c77ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:34.834907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0c73125c-09da-4791-a085-e5b5548c77ea None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:34.836319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0c73125c-09da-4791-a085-e5b5548c77ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:34.900701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1158/2320] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:34.906272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14c30c-3ad9-4f5b-bea5-f22e33f44441 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:34.906599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14c30c-3ad9-4f5b-bea5-f22e33f44441 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:34.906890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e14c30c-3ad9-4f5b-bea5-f22e33f44441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:34.911324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73125c-09da-4791-a085-e5b5548c77ea None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:34.912816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0c73125c-09da-4791-a085-e5b5548c77ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:34.925914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-4e14c30c-3ad9-4f5b-bea5-f22e33f44441 None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:34.927138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4e14c30c-3ad9-4f5b-bea5-f22e33f44441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:34.973638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73125c-09da-4791-a085-e5b5548c77ea None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:34.977268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609ec2a-d8cc-47d1-82b6-93709c5354c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:34.977955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609ec2a-d8cc-47d1-82b6-93709c5354c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:34.978252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609ec2a-d8cc-47d1-82b6-93709c5354c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:34.978551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609ec2a-d8cc-47d1-82b6-93709c5354c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:34.985742 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1163/2321] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:32:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:34.988704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609ec2a-d8cc-47d1-82b6-93709c5354c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:34.990268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609ec2a-d8cc-47d1-82b6-93709c5354c0 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:35.003831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1159/2322] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:35.036491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14c30c-3ad9-4f5b-bea5-f22e33f44441 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:35.037923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4e14c30c-3ad9-4f5b-bea5-f22e33f44441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:35.092066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:35.136125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e14c30c-3ad9-4f5b-bea5-f22e33f44441 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:35.141809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca169b-d70e-444f-8bee-0b15940a25e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:35.142450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca169b-d70e-444f-8bee-0b15940a25e8 service swift] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:35.142758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca169b-d70e-444f-8bee-0b15940a25e8 service swift] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:35.143027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca169b-d70e-444f-8bee-0b15940a25e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:35.151469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca169b-d70e-444f-8bee-0b15940a25e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:35.153269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca169b-d70e-444f-8bee-0b15940a25e8 service swift] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:35.163297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1164/2323] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:32:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:32:35.316104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609ec2a-d8cc-47d1-82b6-93709c5354c0 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:35.319884 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-4609ec2a-d8cc-47d1-82b6-93709c5354c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:35.362965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609ec2a-d8cc-47d1-82b6-93709c5354c0 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:35.368716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:35.369650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:35.369994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:35.370342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] PATH_INFO: `/v3/projects/87500b5c68f54daeb650300bc8d485e8` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:35.372270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87500b5c68f54daeb650300bc8d485e8)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:35.374869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:35.416445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:35.443506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87500b5c68f54daeb650300bc8d485e8', 'request_id': 'req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:35.450502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87500b5c68f54daeb650300bc8d485e8', 'request_id': 'req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:35.461336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87500b5c68f54daeb650300bc8d485e8', 'request_id': 'req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:35.489030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] The token cache is being invalidate because project 87500b5c68f54daeb650300bc8d485e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:32:35.489356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:35.490220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:35.495378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1160/2324] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:32:35 2022] DELETE /identity/v3/projects/87500b5c68f54daeb650300bc8d485e8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:35.496217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca169b-d70e-444f-8bee-0b15940a25e8 service swift] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:35.526783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca169b-d70e-444f-8bee-0b15940a25e8 service swift] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:35.648248 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:35.650943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:35.651361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:35.652261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] PATH_INFO: `/v3/users/cc2a6ce885e24fac89c74f07755683a7` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:35.654106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2a6ce885e24fac89c74f07755683a7)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:35.656704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:35.727477 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2a6ce885e24fac89c74f07755683a7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:35.745263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2a6ce885e24fac89c74f07755683a7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:35.746732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2a6ce885e24fac89c74f07755683a7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:35.755567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2a6ce885e24fac89c74f07755683a7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:35.756323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2a6ce885e24fac89c74f07755683a7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:35.771347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1165/2325] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:32:35 2022] DELETE /identity/v3/users/cc2a6ce885e24fac89c74f07755683a7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:32:36.128824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:36.130929 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:36.148398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b6bce-f2e6-4b77-8d01-90c4242b1f35 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:36.242143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:36.243451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:36.243736 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:36.244018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] PATH_INFO: `/v3/projects/78101a62bbb1443c8b36470a57ce9d10` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:36.245958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] RBAC: Authorizing `identity:delete_project(project_id=78101a62bbb1443c8b36470a57ce9d10)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:36.249161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:36.311269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78101a62bbb1443c8b36470a57ce9d10', 'request_id': 'req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:36.317426 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78101a62bbb1443c8b36470a57ce9d10', 'request_id': 'req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:36.327768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78101a62bbb1443c8b36470a57ce9d10', 'request_id': 'req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:36.344325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] The token cache is being invalidate because project 78101a62bbb1443c8b36470a57ce9d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:32:36.344630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:36.345280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:36.350205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1161/2326] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:32:36 2022] DELETE /identity/v3/projects/78101a62bbb1443c8b36470a57ce9d10 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:36.351378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:36.353059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:36.368701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f55c3e-99de-4a47-94de-7e148a57a9d6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:36.437273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:36.462393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf0d2f-867d-4cbd-98d2-ac7ca1d044d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:36.463582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf0d2f-867d-4cbd-98d2-ac7ca1d044d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:36.463864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf0d2f-867d-4cbd-98d2-ac7ca1d044d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:36.464145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf0d2f-867d-4cbd-98d2-ac7ca1d044d6 admin admin] PATH_INFO: `/v3/users/c6a7fba2c87d4ff2b49e01b87f8dfbd1` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:36.465856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf0d2f-867d-4cbd-98d2-ac7ca1d044d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a7fba2c87d4ff2b49e01b87f8dfbd1)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:36.468789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf0d2f-867d-4cbd-98d2-ac7ca1d044d6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:36.536670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7acf0d2f-867d-4cbd-98d2-ac7ca1d044d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a7fba2c87d4ff2b49e01b87f8dfbd1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:36.551452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7acf0d2f-867d-4cbd-98d2-ac7ca1d044d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a7fba2c87d4ff2b49e01b87f8dfbd1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:36.552171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7acf0d2f-867d-4cbd-98d2-ac7ca1d044d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a7fba2c87d4ff2b49e01b87f8dfbd1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:36.560474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7acf0d2f-867d-4cbd-98d2-ac7ca1d044d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a7fba2c87d4ff2b49e01b87f8dfbd1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:36.561729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7acf0d2f-867d-4cbd-98d2-ac7ca1d044d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a7fba2c87d4ff2b49e01b87f8dfbd1'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:36.577827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1166/2327] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:32:36 2022] DELETE /identity/v3/users/c6a7fba2c87d4ff2b49e01b87f8dfbd1 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:36.986203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:36.987801 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:37.021148 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1e397-dddb-4a1b-9ba8-6dc5236c393f admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:37.081901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:37.103385 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:37.104436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:37.104947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:37.105448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] PATH_INFO: `/v3/projects/3218a96a94084f1888d62d5c9eaf2815` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:37.107341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3218a96a94084f1888d62d5c9eaf2815)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:37.109405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:37.170521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3218a96a94084f1888d62d5c9eaf2815', 'request_id': 'req-e181b61c-9b6c-4298-8b47-87e578f2e3b0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:37.179004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3218a96a94084f1888d62d5c9eaf2815', 'request_id': 'req-e181b61c-9b6c-4298-8b47-87e578f2e3b0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:37.186608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3218a96a94084f1888d62d5c9eaf2815', 'request_id': 'req-e181b61c-9b6c-4298-8b47-87e578f2e3b0'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:37.201536 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] The token cache is being invalidate because project 3218a96a94084f1888d62d5c9eaf2815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:32:37.201878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:37.202507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e181b61c-9b6c-4298-8b47-87e578f2e3b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:37.209388 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1162/2328] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:32:36 2022] DELETE /identity/v3/projects/3218a96a94084f1888d62d5c9eaf2815 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:37.213440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a2024-a5c8-4415-b543-089e1f0d11e1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:37.213986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a2024-a5c8-4415-b543-089e1f0d11e1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:37.214085 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a2024-a5c8-4415-b543-089e1f0d11e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:37.236741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ce0a2024-a5c8-4415-b543-089e1f0d11e1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:37.237900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ce0a2024-a5c8-4415-b543-089e1f0d11e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:37.314985 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1167/2329] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:37.323682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab8db7-2604-46fc-b1a9-1f6271871aa2 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:37.323966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab8db7-2604-46fc-b1a9-1f6271871aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:37.324243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab8db7-2604-46fc-b1a9-1f6271871aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:37.345907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-a7ab8db7-2604-46fc-b1a9-1f6271871aa2 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:37.347296 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a7ab8db7-2604-46fc-b1a9-1f6271871aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:37.355964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1163/2330] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:37.362105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a2024-a5c8-4415-b543-089e1f0d11e1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:37.376441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a2024-a5c8-4415-b543-089e1f0d11e1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:37.381517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a02f3a-5923-4b08-b3ec-8ee3d9658e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:37.382385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a02f3a-5923-4b08-b3ec-8ee3d9658e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:37.382775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a02f3a-5923-4b08-b3ec-8ee3d9658e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:37.383134 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a02f3a-5923-4b08-b3ec-8ee3d9658e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:37.383945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a02f3a-5923-4b08-b3ec-8ee3d9658e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:32:37.384322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a02f3a-5923-4b08-b3ec-8ee3d9658e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:37.386027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a02f3a-5923-4b08-b3ec-8ee3d9658e4f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:37.399437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1168/2331] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:32:37.403694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab8db7-2604-46fc-b1a9-1f6271871aa2 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:37.416027 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab8db7-2604-46fc-b1a9-1f6271871aa2 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:37.422417 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ee779-9bc0-49a1-a8f9-040623ac93aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:37.423501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ee779-9bc0-49a1-a8f9-040623ac93aa admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:37.423845 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ee779-9bc0-49a1-a8f9-040623ac93aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:37.424204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ee779-9bc0-49a1-a8f9-040623ac93aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:37.424970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ee779-9bc0-49a1-a8f9-040623ac93aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:37.427050 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ee779-9bc0-49a1-a8f9-040623ac93aa admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:37.462182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1164/2332] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:32:37 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:32:37.463715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a02f3a-5923-4b08-b3ec-8ee3d9658e4f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:37.475663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a02f3a-5923-4b08-b3ec-8ee3d9658e4f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:37.533542 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:37.552107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75052c-91c9-4435-879f-0683f53dc041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:37.552811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75052c-91c9-4435-879f-0683f53dc041 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:37.553091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75052c-91c9-4435-879f-0683f53dc041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:37.553350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c75052c-91c9-4435-879f-0683f53dc041 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:37.554044 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75052c-91c9-4435-879f-0683f53dc041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:37.556044 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c75052c-91c9-4435-879f-0683f53dc041 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:37.582485 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1169/2333] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:32:37 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:32:37.586810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ee779-9bc0-49a1-a8f9-040623ac93aa admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:37.607938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ee779-9bc0-49a1-a8f9-040623ac93aa admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:37.616798 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226aa6e-b80c-4f7e-be9e-9911abc461de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:37.617607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226aa6e-b80c-4f7e-be9e-9911abc461de admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:37.617966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226aa6e-b80c-4f7e-be9e-9911abc461de admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:37.618345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226aa6e-b80c-4f7e-be9e-9911abc461de admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:37.619180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226aa6e-b80c-4f7e-be9e-9911abc461de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:37.620836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226aa6e-b80c-4f7e-be9e-9911abc461de admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:37.637545 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1165/2334] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:37.639457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75052c-91c9-4435-879f-0683f53dc041 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:37.656191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c75052c-91c9-4435-879f-0683f53dc041 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:37.661268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c512e-bd32-422f-a6b7-e73fa9fe0fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:37.661943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c512e-bd32-422f-a6b7-e73fa9fe0fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:37.662232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c512e-bd32-422f-a6b7-e73fa9fe0fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:37.662512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c512e-bd32-422f-a6b7-e73fa9fe0fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:37.663282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c512e-bd32-422f-a6b7-e73fa9fe0fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:37.664972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c512e-bd32-422f-a6b7-e73fa9fe0fe1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:37.682564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1170/2335] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:37.685845 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226aa6e-b80c-4f7e-be9e-9911abc461de admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:37.697037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226aa6e-b80c-4f7e-be9e-9911abc461de admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:37.704820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc10ebf-4f96-4f26-a523-c99e6d927e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:37.705715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc10ebf-4f96-4f26-a523-c99e6d927e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:37.705995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc10ebf-4f96-4f26-a523-c99e6d927e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:37.706271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc10ebf-4f96-4f26-a523-c99e6d927e84 admin admin] PATH_INFO: `/v3/projects/bb3468380acf4a7c8062a16ba19144a6/users/4c0090dffda841a49739955b68c477b4/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:37.728391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc10ebf-4f96-4f26-a523-c99e6d927e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3468380acf4a7c8062a16ba19144a6, user_id=4c0090dffda841a49739955b68c477b4, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:37.731436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc10ebf-4f96-4f26-a523-c99e6d927e84 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:37.750769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1166/2336] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:32:37 2022] PUT /identity/v3/projects/bb3468380acf4a7c8062a16ba19144a6/users/4c0090dffda841a49739955b68c477b4/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:37.766851 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121f2d8-97df-47bb-97d8-479f477c18de None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:37.767378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121f2d8-97df-47bb-97d8-479f477c18de None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:37.767786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121f2d8-97df-47bb-97d8-479f477c18de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:37.807123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a121f2d8-97df-47bb-97d8-479f477c18de None None] MFA Rules not processed for user `4c0090dffda841a49739955b68c477b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:37.808396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a121f2d8-97df-47bb-97d8-479f477c18de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:37.911924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1171/2337] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:37.918878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc10ebf-4f96-4f26-a523-c99e6d927e84 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:37.996466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc10ebf-4f96-4f26-a523-c99e6d927e84 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:38.003683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46e6a1-7593-4649-896c-cc67efe70ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:38.004688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46e6a1-7593-4649-896c-cc67efe70ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.005149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46e6a1-7593-4649-896c-cc67efe70ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.005726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46e6a1-7593-4649-896c-cc67efe70ef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.006644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46e6a1-7593-4649-896c-cc67efe70ef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:38.008848 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46e6a1-7593-4649-896c-cc67efe70ef6 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:38.068838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:38.076201 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1167/2338] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:32:37 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:32:38.081897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121f2d8-97df-47bb-97d8-479f477c18de None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:38.093504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121f2d8-97df-47bb-97d8-479f477c18de None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:38.171829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23d0de-6ed6-43d5-9155-7291718cad13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:38.172518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23d0de-6ed6-43d5-9155-7291718cad13 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.172802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23d0de-6ed6-43d5-9155-7291718cad13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.173078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23d0de-6ed6-43d5-9155-7291718cad13 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.173892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23d0de-6ed6-43d5-9155-7291718cad13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:38.175920 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23d0de-6ed6-43d5-9155-7291718cad13 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:38.202874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1172/2339] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:32:38 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:32:38.207049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46e6a1-7593-4649-896c-cc67efe70ef6 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:38.219797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46e6a1-7593-4649-896c-cc67efe70ef6 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:38.225079 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36866866-f7e2-4e02-b8fd-cb9df7771635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:38.225847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36866866-f7e2-4e02-b8fd-cb9df7771635 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.226226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36866866-f7e2-4e02-b8fd-cb9df7771635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.226597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36866866-f7e2-4e02-b8fd-cb9df7771635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.227399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36866866-f7e2-4e02-b8fd-cb9df7771635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:38.229055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36866866-f7e2-4e02-b8fd-cb9df7771635 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:38.245875 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1168/2340] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:38.247126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23d0de-6ed6-43d5-9155-7291718cad13 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:38.258245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23d0de-6ed6-43d5-9155-7291718cad13 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:38.266796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8600a90-a0eb-463c-b1ed-228627a22de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:38.267488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600a90-a0eb-463c-b1ed-228627a22de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.267767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600a90-a0eb-463c-b1ed-228627a22de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.268045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8600a90-a0eb-463c-b1ed-228627a22de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.268780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8600a90-a0eb-463c-b1ed-228627a22de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:38.270590 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8600a90-a0eb-463c-b1ed-228627a22de3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:38.281686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1173/2341] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:38.286073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36866866-f7e2-4e02-b8fd-cb9df7771635 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:38.297453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36866866-f7e2-4e02-b8fd-cb9df7771635 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:38.304938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1bbd2-bdf3-432d-b8a6-48c9c3592189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:38.305867 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1bbd2-bdf3-432d-b8a6-48c9c3592189 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.306165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1bbd2-bdf3-432d-b8a6-48c9c3592189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.306440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1bbd2-bdf3-432d-b8a6-48c9c3592189 admin admin] PATH_INFO: `/v3/projects/0ce25bc8a2d44360912281f519df7ed4/users/e0212930096c425b91c10ddcb2a690ab/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.322651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1bbd2-bdf3-432d-b8a6-48c9c3592189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce25bc8a2d44360912281f519df7ed4, user_id=e0212930096c425b91c10ddcb2a690ab, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:38.327160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1bbd2-bdf3-432d-b8a6-48c9c3592189 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:38.343933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1169/2342] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:32:38 2022] PUT /identity/v3/projects/0ce25bc8a2d44360912281f519df7ed4/users/e0212930096c425b91c10ddcb2a690ab/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:38.355829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62ba7-7654-411c-8597-8ffcc2056350 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.356160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62ba7-7654-411c-8597-8ffcc2056350 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.356392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c62ba7-7654-411c-8597-8ffcc2056350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.388361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b6c62ba7-7654-411c-8597-8ffcc2056350 None None] MFA Rules not processed for user `e0212930096c425b91c10ddcb2a690ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:38.389583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b6c62ba7-7654-411c-8597-8ffcc2056350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:38.418415 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:38.465032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1174/2343] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:38.470204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1bbd2-bdf3-432d-b8a6-48c9c3592189 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:38.538352 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1bbd2-bdf3-432d-b8a6-48c9c3592189 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:38.543506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a6112-0df6-4778-8e35-3bd9ee9e2147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:38.544915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a6112-0df6-4778-8e35-3bd9ee9e2147 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.545268 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a6112-0df6-4778-8e35-3bd9ee9e2147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.545615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a6112-0df6-4778-8e35-3bd9ee9e2147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.546392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a6112-0df6-4778-8e35-3bd9ee9e2147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:38.548428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a6112-0df6-4778-8e35-3bd9ee9e2147 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:38.577495 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1170/2344] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:32:38 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:32:38.581109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c62ba7-7654-411c-8597-8ffcc2056350 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:38.595292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c62ba7-7654-411c-8597-8ffcc2056350 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:38.675280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82001cc-4a28-465b-a1b2-4ce44ef88336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:38.676405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82001cc-4a28-465b-a1b2-4ce44ef88336 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.676850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82001cc-4a28-465b-a1b2-4ce44ef88336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.677473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82001cc-4a28-465b-a1b2-4ce44ef88336 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.678178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82001cc-4a28-465b-a1b2-4ce44ef88336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:38.680858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82001cc-4a28-465b-a1b2-4ce44ef88336 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:38.704751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1175/2345] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:32:38 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:32:38.708920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a6112-0df6-4778-8e35-3bd9ee9e2147 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:38.721520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a6112-0df6-4778-8e35-3bd9ee9e2147 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:38.728574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ac4a6-2143-4452-9c55-345606f6e643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:38.729251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ac4a6-2143-4452-9c55-345606f6e643 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.729538 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ac4a6-2143-4452-9c55-345606f6e643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.729820 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039ac4a6-2143-4452-9c55-345606f6e643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.730597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ac4a6-2143-4452-9c55-345606f6e643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:38.733345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039ac4a6-2143-4452-9c55-345606f6e643 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:38.746375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1171/2346] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:38.750606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82001cc-4a28-465b-a1b2-4ce44ef88336 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:38.761643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82001cc-4a28-465b-a1b2-4ce44ef88336 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:38.766815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011b0ec-8cf3-4489-ba44-7f995aff4243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:38.767719 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011b0ec-8cf3-4489-ba44-7f995aff4243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.768016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011b0ec-8cf3-4489-ba44-7f995aff4243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.768297 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011b0ec-8cf3-4489-ba44-7f995aff4243 admin admin] PATH_INFO: `/v3/projects/ffbecd1ef5434c33ade3a0c382f4fdb7/users/eb97963b0d404c169d89647fabfe3d1d/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.785787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011b0ec-8cf3-4489-ba44-7f995aff4243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffbecd1ef5434c33ade3a0c382f4fdb7, user_id=eb97963b0d404c169d89647fabfe3d1d, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:38.788531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f011b0ec-8cf3-4489-ba44-7f995aff4243 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:38.806496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1176/2347] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:32:38 2022] PUT /identity/v3/projects/ffbecd1ef5434c33ade3a0c382f4fdb7/users/eb97963b0d404c169d89647fabfe3d1d/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:38.810614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ac4a6-2143-4452-9c55-345606f6e643 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:38.821693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039ac4a6-2143-4452-9c55-345606f6e643 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:38.887334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758d707-0784-4020-a472-56072cbbec92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:38.888827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758d707-0784-4020-a472-56072cbbec92 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.889110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758d707-0784-4020-a472-56072cbbec92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.889400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758d707-0784-4020-a472-56072cbbec92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.890118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758d707-0784-4020-a472-56072cbbec92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:38.891882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758d707-0784-4020-a472-56072cbbec92 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:38.905564 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1172/2348] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:38.908612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011b0ec-8cf3-4489-ba44-7f995aff4243 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:38.922665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011b0ec-8cf3-4489-ba44-7f995aff4243 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:38.928324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8468a-6821-4d21-8b6e-ae0cb7232da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:38.929419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8468a-6821-4d21-8b6e-ae0cb7232da0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.929893 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8468a-6821-4d21-8b6e-ae0cb7232da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.930175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f8468a-6821-4d21-8b6e-ae0cb7232da0 admin admin] PATH_INFO: `/v3/projects/ffbecd1ef5434c33ade3a0c382f4fdb7/users/eb97963b0d404c169d89647fabfe3d1d/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.932263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8468a-6821-4d21-8b6e-ae0cb7232da0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffbecd1ef5434c33ade3a0c382f4fdb7, user_id=eb97963b0d404c169d89647fabfe3d1d, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:38.935029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f8468a-6821-4d21-8b6e-ae0cb7232da0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:38.949209 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1177/2349] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:32:38 2022] PUT /identity/v3/projects/ffbecd1ef5434c33ade3a0c382f4fdb7/users/eb97963b0d404c169d89647fabfe3d1d/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:38.960168 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cde93-2012-4745-a9bb-7e72bbb214d0 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:38.960465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cde93-2012-4745-a9bb-7e72bbb214d0 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:38.963161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cde93-2012-4745-a9bb-7e72bbb214d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:38.992162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-1a3cde93-2012-4745-a9bb-7e72bbb214d0 None None] MFA Rules not processed for user `eb97963b0d404c169d89647fabfe3d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:38.993386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1a3cde93-2012-4745-a9bb-7e72bbb214d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:39.018270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:39.081991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1173/2350] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:39.094395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8468a-6821-4d21-8b6e-ae0cb7232da0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:39.096718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-36f8468a-6821-4d21-8b6e-ae0cb7232da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:39.177230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f8468a-6821-4d21-8b6e-ae0cb7232da0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:39.180909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b78127-39b6-44e1-92c9-927ba7f5c48f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:39.181576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b78127-39b6-44e1-92c9-927ba7f5c48f service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:39.181849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b78127-39b6-44e1-92c9-927ba7f5c48f service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:39.182120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b78127-39b6-44e1-92c9-927ba7f5c48f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:39.222780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:39.258105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b78127-39b6-44e1-92c9-927ba7f5c48f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:39.259802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b78127-39b6-44e1-92c9-927ba7f5c48f service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:39.270127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1178/2351] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:40.835302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cde93-2012-4745-a9bb-7e72bbb214d0 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:40.836796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-1a3cde93-2012-4745-a9bb-7e72bbb214d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:40.850624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cde93-2012-4745-a9bb-7e72bbb214d0 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:40.854446 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9304ccf-a1f0-4eb2-add2-696dd721c048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:40.855160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9304ccf-a1f0-4eb2-add2-696dd721c048 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:40.855437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9304ccf-a1f0-4eb2-add2-696dd721c048 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:40.855716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9304ccf-a1f0-4eb2-add2-696dd721c048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:40.957959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9304ccf-a1f0-4eb2-add2-696dd721c048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:40.959644 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9304ccf-a1f0-4eb2-add2-696dd721c048 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:40.968763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1174/2352] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:41.605870 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b78127-39b6-44e1-92c9-927ba7f5c48f service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:41.615376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b78127-39b6-44e1-92c9-927ba7f5c48f service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:41.623496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ccca9-b721-4f40-b398-34119e29e07b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:41.624414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ccca9-b721-4f40-b398-34119e29e07b service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:41.624693 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ccca9-b721-4f40-b398-34119e29e07b service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:41.624963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ccca9-b721-4f40-b398-34119e29e07b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:41.635207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ccca9-b721-4f40-b398-34119e29e07b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:41.639254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ccca9-b721-4f40-b398-34119e29e07b service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:41.659508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1179/2353] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:45.641530 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9304ccf-a1f0-4eb2-add2-696dd721c048 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:45.708047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9304ccf-a1f0-4eb2-add2-696dd721c048 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:45.712501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:45.713354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:45.713632 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:45.713924 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] PATH_INFO: `/v3/users/4c0090dffda841a49739955b68c477b4` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:45.715127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0090dffda841a49739955b68c477b4)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:45.717087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:45.763747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0090dffda841a49739955b68c477b4'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:45.771351 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:45.776383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0090dffda841a49739955b68c477b4'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:45.777240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0090dffda841a49739955b68c477b4'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:45.784315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0090dffda841a49739955b68c477b4'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:45.784948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0090dffda841a49739955b68c477b4'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:45.798359 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1175/2354] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:32:45 2022] DELETE /identity/v3/users/4c0090dffda841a49739955b68c477b4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:45.803133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760c471-0816-4a0f-857b-11f8743f34c5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:45.804016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760c471-0816-4a0f-857b-11f8743f34c5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:45.804016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760c471-0816-4a0f-857b-11f8743f34c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:45.824018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6760c471-0816-4a0f-857b-11f8743f34c5 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:45.825290 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6760c471-0816-4a0f-857b-11f8743f34c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:45.885593 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1180/2355] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:45.895971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:45.897395 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:45.954254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84469dc2-bef7-4946-bf25-30ca19c5cd1b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:45.957812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83daecf-18bc-469c-81db-3279883a3b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:45.958457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83daecf-18bc-469c-81db-3279883a3b9b service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:45.958769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83daecf-18bc-469c-81db-3279883a3b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:45.959046 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83daecf-18bc-469c-81db-3279883a3b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:45.968647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83daecf-18bc-469c-81db-3279883a3b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:45.970075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83daecf-18bc-469c-81db-3279883a3b9b service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:45.979401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1176/2356] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:46.269624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760c471-0816-4a0f-857b-11f8743f34c5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:46.271550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6760c471-0816-4a0f-857b-11f8743f34c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:46.288625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760c471-0816-4a0f-857b-11f8743f34c5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:46.295867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:46.296704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:46.296965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:46.297381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] PATH_INFO: `/v3/projects/bb3468380acf4a7c8062a16ba19144a6` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:46.298437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb3468380acf4a7c8062a16ba19144a6)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:46.300547 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:46.350423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb3468380acf4a7c8062a16ba19144a6', 'request_id': 'req-2c0f40e6-7e13-4f0c-b629-664b7e26c686'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.357166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb3468380acf4a7c8062a16ba19144a6', 'request_id': 'req-2c0f40e6-7e13-4f0c-b629-664b7e26c686'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.365443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb3468380acf4a7c8062a16ba19144a6', 'request_id': 'req-2c0f40e6-7e13-4f0c-b629-664b7e26c686'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.377896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] The token cache is being invalidate because project bb3468380acf4a7c8062a16ba19144a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:32:46.378187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.378842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.380649 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1181/2357] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:32:46 2022] DELETE /identity/v3/projects/bb3468380acf4a7c8062a16ba19144a6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:46.384340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83daecf-18bc-469c-81db-3279883a3b9b service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:46.395092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83daecf-18bc-469c-81db-3279883a3b9b service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:46.453950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:46.454852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:46.455140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:46.455416 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] PATH_INFO: `/v3/users/e0212930096c425b91c10ddcb2a690ab` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:46.456607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0212930096c425b91c10ddcb2a690ab)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:46.458599 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:46.505373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0212930096c425b91c10ddcb2a690ab'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.518873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0212930096c425b91c10ddcb2a690ab'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.519647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0212930096c425b91c10ddcb2a690ab'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.526657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0212930096c425b91c10ddcb2a690ab'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.527294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0212930096c425b91c10ddcb2a690ab'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.536248 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:46.542773 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1177/2358] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:32:46 2022] DELETE /identity/v3/users/e0212930096c425b91c10ddcb2a690ab => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:46.824774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:46.826359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:46.839875 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0f40e6-7e13-4f0c-b629-664b7e26c686 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:46.890326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:46.908998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:46.909865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:46.910149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:46.910427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] PATH_INFO: `/v3/projects/0ce25bc8a2d44360912281f519df7ed4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:46.911647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce25bc8a2d44360912281f519df7ed4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:46.913571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:46.958773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce25bc8a2d44360912281f519df7ed4', 'request_id': 'req-647ddda3-1e56-4f7c-b1a6-27c20d1551db'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.964306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce25bc8a2d44360912281f519df7ed4', 'request_id': 'req-647ddda3-1e56-4f7c-b1a6-27c20d1551db'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.972380 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce25bc8a2d44360912281f519df7ed4', 'request_id': 'req-647ddda3-1e56-4f7c-b1a6-27c20d1551db'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.984083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] The token cache is being invalidate because project 0ce25bc8a2d44360912281f519df7ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:32:46.984370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.985000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:46.986582 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1182/2359] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:32:46 2022] DELETE /identity/v3/projects/0ce25bc8a2d44360912281f519df7ed4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:46.990560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:46.991986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:47.004441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b722ba-2cae-48eb-becc-61ba37d76c2d admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:47.059711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af22b7-f287-4711-9439-d80bf2d3dba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:47.060480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af22b7-f287-4711-9439-d80bf2d3dba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:47.060761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af22b7-f287-4711-9439-d80bf2d3dba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:47.061047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af22b7-f287-4711-9439-d80bf2d3dba7 admin admin] PATH_INFO: `/v3/users/eb97963b0d404c169d89647fabfe3d1d` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:47.062505 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af22b7-f287-4711-9439-d80bf2d3dba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb97963b0d404c169d89647fabfe3d1d)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:47.064590 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af22b7-f287-4711-9439-d80bf2d3dba7 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:47.111767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b1af22b7-f287-4711-9439-d80bf2d3dba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb97963b0d404c169d89647fabfe3d1d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:47.123524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b1af22b7-f287-4711-9439-d80bf2d3dba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb97963b0d404c169d89647fabfe3d1d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:47.124175 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b1af22b7-f287-4711-9439-d80bf2d3dba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb97963b0d404c169d89647fabfe3d1d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:47.131065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b1af22b7-f287-4711-9439-d80bf2d3dba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb97963b0d404c169d89647fabfe3d1d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:47.131689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-b1af22b7-f287-4711-9439-d80bf2d3dba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb97963b0d404c169d89647fabfe3d1d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:47.145530 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1178/2360] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:32:46 2022] DELETE /identity/v3/users/eb97963b0d404c169d89647fabfe3d1d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:32:47.415523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:47.416957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:47.429903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ddda3-1e56-4f7c-b1a6-27c20d1551db admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:47.493836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:47.494745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:47.495024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:47.495320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] PATH_INFO: `/v3/projects/ffbecd1ef5434c33ade3a0c382f4fdb7` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:47.496480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffbecd1ef5434c33ade3a0c382f4fdb7)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:47.498558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:47.547688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffbecd1ef5434c33ade3a0c382f4fdb7', 'request_id': 'req-4117bd19-aa7b-4538-aa2c-f196af5393e2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:47.553506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffbecd1ef5434c33ade3a0c382f4fdb7', 'request_id': 'req-4117bd19-aa7b-4538-aa2c-f196af5393e2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:47.561289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffbecd1ef5434c33ade3a0c382f4fdb7', 'request_id': 'req-4117bd19-aa7b-4538-aa2c-f196af5393e2'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:47.573267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] The token cache is being invalidate because project ffbecd1ef5434c33ade3a0c382f4fdb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:32:47.573883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:47.574766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4117bd19-aa7b-4538-aa2c-f196af5393e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:32:47.581261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1183/2361] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:32:47 2022] DELETE /identity/v3/projects/ffbecd1ef5434c33ade3a0c382f4fdb7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:47.586303 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66393c6-fc84-4526-8a3f-6eb71689cf10 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:47.586636 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66393c6-fc84-4526-8a3f-6eb71689cf10 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:47.586926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66393c6-fc84-4526-8a3f-6eb71689cf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:47.607233 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-b66393c6-fc84-4526-8a3f-6eb71689cf10 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:47.608700 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b66393c6-fc84-4526-8a3f-6eb71689cf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:47.670862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1179/2362] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:47.678977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fce54-6847-403c-92ee-3cb86038373a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:47.679267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fce54-6847-403c-92ee-3cb86038373a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:47.679548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fce54-6847-403c-92ee-3cb86038373a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:47.685997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:47.701776 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c70fce54-6847-403c-92ee-3cb86038373a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:47.703024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c70fce54-6847-403c-92ee-3cb86038373a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:47.710929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1184/2363] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:47.715355 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66393c6-fc84-4526-8a3f-6eb71689cf10 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:47.725677 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66393c6-fc84-4526-8a3f-6eb71689cf10 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:47.730347 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f72f4d-d595-41f1-933e-9a425499a521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:47.731032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f72f4d-d595-41f1-933e-9a425499a521 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:47.732004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f72f4d-d595-41f1-933e-9a425499a521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:47.732286 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f72f4d-d595-41f1-933e-9a425499a521 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:47.733058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f72f4d-d595-41f1-933e-9a425499a521 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:32:47.733368 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f72f4d-d595-41f1-933e-9a425499a521 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:47.735012 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f72f4d-d595-41f1-933e-9a425499a521 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:47.746710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1180/2364] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:32:47.751200 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fce54-6847-403c-92ee-3cb86038373a None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:47.760349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fce54-6847-403c-92ee-3cb86038373a None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:47.766510 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca2a68-b934-4966-9ae0-2799e7aead65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:47.767229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca2a68-b934-4966-9ae0-2799e7aead65 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:47.767645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca2a68-b934-4966-9ae0-2799e7aead65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:47.768014 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca2a68-b934-4966-9ae0-2799e7aead65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:47.768820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca2a68-b934-4966-9ae0-2799e7aead65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:47.770867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca2a68-b934-4966-9ae0-2799e7aead65 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:47.812716 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1185/2365] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:32:47 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:32:47.813928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f72f4d-d595-41f1-933e-9a425499a521 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:47.820709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:47.825759 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f72f4d-d595-41f1-933e-9a425499a521 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:47.887993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817bbc4-f39c-4521-a5c1-0be5479e0ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:47.888689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817bbc4-f39c-4521-a5c1-0be5479e0ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:47.888972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817bbc4-f39c-4521-a5c1-0be5479e0ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:47.889243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817bbc4-f39c-4521-a5c1-0be5479e0ccc admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:47.889929 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817bbc4-f39c-4521-a5c1-0be5479e0ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:47.891907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817bbc4-f39c-4521-a5c1-0be5479e0ccc admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:47.912140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1181/2366] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:32:47 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:32:47.916589 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca2a68-b934-4966-9ae0-2799e7aead65 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:47.927503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca2a68-b934-4966-9ae0-2799e7aead65 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:47.932170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d116c6c-daa8-4714-bcc0-1e979c514266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:47.932829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116c6c-daa8-4714-bcc0-1e979c514266 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:47.933144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116c6c-daa8-4714-bcc0-1e979c514266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:47.933421 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116c6c-daa8-4714-bcc0-1e979c514266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:47.934137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d116c6c-daa8-4714-bcc0-1e979c514266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:47.936316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d116c6c-daa8-4714-bcc0-1e979c514266 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:47.946290 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1186/2367] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:47.950438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817bbc4-f39c-4521-a5c1-0be5479e0ccc admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:47.960400 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817bbc4-f39c-4521-a5c1-0be5479e0ccc admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:47.965105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094f19-054a-4f1e-b9f4-b1dfa7a8ed21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:47.965780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094f19-054a-4f1e-b9f4-b1dfa7a8ed21 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:47.966057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094f19-054a-4f1e-b9f4-b1dfa7a8ed21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:47.966330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094f19-054a-4f1e-b9f4-b1dfa7a8ed21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:47.967082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094f19-054a-4f1e-b9f4-b1dfa7a8ed21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:47.968703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094f19-054a-4f1e-b9f4-b1dfa7a8ed21 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:47.980409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1182/2368] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:32:47.985051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d116c6c-daa8-4714-bcc0-1e979c514266 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:47.995090 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d116c6c-daa8-4714-bcc0-1e979c514266 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:48.000316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894808e-2e07-4bc4-b98e-163da6dfd72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:48.001184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894808e-2e07-4bc4-b98e-163da6dfd72d admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:48.001471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894808e-2e07-4bc4-b98e-163da6dfd72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:48.001747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894808e-2e07-4bc4-b98e-163da6dfd72d admin admin] PATH_INFO: `/v3/projects/9b8730afcac7420baa2d1f69526234af/users/252fd21e3de9449bae5bdcd3e4e4fc1c/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:48.016829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894808e-2e07-4bc4-b98e-163da6dfd72d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b8730afcac7420baa2d1f69526234af, user_id=252fd21e3de9449bae5bdcd3e4e4fc1c, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:48.019645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894808e-2e07-4bc4-b98e-163da6dfd72d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:48.032189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1187/2369] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:32:47 2022] PUT /identity/v3/projects/9b8730afcac7420baa2d1f69526234af/users/252fd21e3de9449bae5bdcd3e4e4fc1c/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:32:48.043320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24f84d-dfab-4e8d-8ddd-bcb0611a235a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:48.044361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24f84d-dfab-4e8d-8ddd-bcb0611a235a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:48.044457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24f84d-dfab-4e8d-8ddd-bcb0611a235a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:48.070252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0b24f84d-dfab-4e8d-8ddd-bcb0611a235a None None] MFA Rules not processed for user `252fd21e3de9449bae5bdcd3e4e4fc1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:48.071492 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0b24f84d-dfab-4e8d-8ddd-bcb0611a235a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:48.134895 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1183/2370] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:32:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:48.147007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894808e-2e07-4bc4-b98e-163da6dfd72d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:48.148690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0894808e-2e07-4bc4-b98e-163da6dfd72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:48.206638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894808e-2e07-4bc4-b98e-163da6dfd72d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:48.210311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80bc16-fbab-4027-88f6-1b07fa2094f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:48.210989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80bc16-fbab-4027-88f6-1b07fa2094f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:48.211270 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80bc16-fbab-4027-88f6-1b07fa2094f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:48.211546 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80bc16-fbab-4027-88f6-1b07fa2094f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:48.220317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80bc16-fbab-4027-88f6-1b07fa2094f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:48.221741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80bc16-fbab-4027-88f6-1b07fa2094f2 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:48.232070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1188/2371] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:49.615231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303586-79a8-463a-9d2d-67dd4bc88aa1 None None] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:49.615782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303586-79a8-463a-9d2d-67dd4bc88aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:49.615782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41303586-79a8-463a-9d2d-67dd4bc88aa1 None None] PATH_INFO: `/` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:49.617133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1184/2372] 10.222.0.93 () {54 vars in 921 bytes} [Fri Jun 3 15:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 03 15:32:49.622475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4a836-e537-47ae-ab36-2ee302551ed6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:49.622987 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4a836-e537-47ae-ab36-2ee302551ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:49.623083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4a836-e537-47ae-ab36-2ee302551ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:49.643957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-99d4a836-e537-47ae-ab36-2ee302551ed6 None None] MFA Rules not processed for user `c1966524091242e7a6cd2758027a0d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:49.646027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-99d4a836-e537-47ae-ab36-2ee302551ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:49.728757 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1189/2373] 10.222.0.93 () {58 vars in 1018 bytes} [Fri Jun 3 15:32:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:49.744327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41303586-79a8-463a-9d2d-67dd4bc88aa1 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:49.747273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-41303586-79a8-463a-9d2d-67dd4bc88aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:49.810217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41303586-79a8-463a-9d2d-67dd4bc88aa1 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:49.813866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c25b17-a095-4a56-be33-3bb1849cd328 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13121a263c31433dbd4a5082b4bf3bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:49.814523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c25b17-a095-4a56-be33-3bb1849cd328 service glance] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:49.814911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c25b17-a095-4a56-be33-3bb1849cd328 service glance] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:49.815106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c25b17-a095-4a56-be33-3bb1849cd328 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:49.834619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c25b17-a095-4a56-be33-3bb1849cd328 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:49.836136 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c25b17-a095-4a56-be33-3bb1849cd328 service glance] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:49.847212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1185/2374] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:32:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:32:51.730224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f196f72-e477-4bbc-9dbe-4d2adce3abf1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:51.730765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f196f72-e477-4bbc-9dbe-4d2adce3abf1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:51.730882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f196f72-e477-4bbc-9dbe-4d2adce3abf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:51.748308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1f196f72-e477-4bbc-9dbe-4d2adce3abf1 None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:32:51.749487 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1f196f72-e477-4bbc-9dbe-4d2adce3abf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:51.767973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:51.804837 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1190/2375] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:32:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:32:51.816504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c25b17-a095-4a56-be33-3bb1849cd328 service glance] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:32:51.817943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-49c25b17-a095-4a56-be33-3bb1849cd328 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:32:51.836727 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:32:51.889149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c25b17-a095-4a56-be33-3bb1849cd328 service glance] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:32:51.894143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb3009-9882-4b50-972e-d66f572e5952 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:32:51.894821 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb3009-9882-4b50-972e-d66f572e5952 service swift] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:32:51.895108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb3009-9882-4b50-972e-d66f572e5952 service swift] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:32:51.895386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb3009-9882-4b50-972e-d66f572e5952 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:32:51.904098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb3009-9882-4b50-972e-d66f572e5952 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:32:51.905718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb3009-9882-4b50-972e-d66f572e5952 service swift] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:32:51.914185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1186/2376] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:32:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:33:01.798044 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=14 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:01.910542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=25 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:08.281266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbb0ba-7865-45da-acf8-72a818832458 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:08.283183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbb0ba-7865-45da-acf8-72a818832458 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:08.283183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbb0ba-7865-45da-acf8-72a818832458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:08.302383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-78dbb0ba-7865-45da-acf8-72a818832458 None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:08.303626 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-78dbb0ba-7865-45da-acf8-72a818832458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:08.310958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1191/2377] 10.222.0.93 () {58 vars in 967 bytes} [Fri Jun 3 15:33:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:08.324508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb3009-9882-4b50-972e-d66f572e5952 service swift] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:08.336006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb3009-9882-4b50-972e-d66f572e5952 service swift] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:08.341135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b155d49-cb80-4f89-8844-c6bc469e6455 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:08.341761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155d49-cb80-4f89-8844-c6bc469e6455 service swift] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:08.342043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155d49-cb80-4f89-8844-c6bc469e6455 service swift] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:08.342360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b155d49-cb80-4f89-8844-c6bc469e6455 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:08.351162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b155d49-cb80-4f89-8844-c6bc469e6455 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:08.352735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b155d49-cb80-4f89-8844-c6bc469e6455 service swift] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:08.361167 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1187/2378] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:33:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:33:18.296503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:18.360102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=6 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:26.503818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbb0ba-7865-45da-acf8-72a818832458 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:26.567425 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbb0ba-7865-45da-acf8-72a818832458 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:26.572007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:26.572857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:26.573135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:26.573406 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] PATH_INFO: `/v3/users/252fd21e3de9449bae5bdcd3e4e4fc1c` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:26.574633 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=252fd21e3de9449bae5bdcd3e4e4fc1c)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:26.576555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:26.621076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '252fd21e3de9449bae5bdcd3e4e4fc1c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:26.632732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '252fd21e3de9449bae5bdcd3e4e4fc1c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:26.633495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '252fd21e3de9449bae5bdcd3e4e4fc1c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:26.640417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '252fd21e3de9449bae5bdcd3e4e4fc1c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:26.641030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '252fd21e3de9449bae5bdcd3e4e4fc1c'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:26.653938 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1192/2379] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:33:26 2022] DELETE /identity/v3/users/252fd21e3de9449bae5bdcd3e4e4fc1c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:26.658956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89f079-f021-468a-8ffc-ed797fd731bd None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:26.659436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89f079-f021-468a-8ffc-ed797fd731bd None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:26.659555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89f079-f021-468a-8ffc-ed797fd731bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:26.678338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-8c89f079-f021-468a-8ffc-ed797fd731bd None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:26.679635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8c89f079-f021-468a-8ffc-ed797fd731bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:26.738047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1188/2380] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:26.747816 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:26.749234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:26.806223 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246997e5-2538-4c1b-be26-3fdb1def8caf admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:26.809811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff3fcc-e3eb-4442-94ee-65c2d904fa29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:26.810507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff3fcc-e3eb-4442-94ee-65c2d904fa29 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:26.810803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff3fcc-e3eb-4442-94ee-65c2d904fa29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:26.811072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff3fcc-e3eb-4442-94ee-65c2d904fa29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:26.819836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff3fcc-e3eb-4442-94ee-65c2d904fa29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:26.821590 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff3fcc-e3eb-4442-94ee-65c2d904fa29 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:26.831325 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1193/2381] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:33:27.102152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89f079-f021-468a-8ffc-ed797fd731bd None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:27.103735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-8c89f079-f021-468a-8ffc-ed797fd731bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:27.115832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89f079-f021-468a-8ffc-ed797fd731bd None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:27.120547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:27.121413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:27.121699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:27.121980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] PATH_INFO: `/v3/projects/9b8730afcac7420baa2d1f69526234af` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:27.123191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b8730afcac7420baa2d1f69526234af)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:27.125621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:27.175324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b8730afcac7420baa2d1f69526234af', 'request_id': 'req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:27.180695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b8730afcac7420baa2d1f69526234af', 'request_id': 'req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:27.188943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b8730afcac7420baa2d1f69526234af', 'request_id': 'req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:27.200749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] The token cache is being invalidate because project 9b8730afcac7420baa2d1f69526234af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:33:27.201163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:27.201930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-8bc3c94f-b0ed-41cc-b8ae-84ac03fa0d20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:27.204468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1189/2382] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:33:27 2022] DELETE /identity/v3/projects/9b8730afcac7420baa2d1f69526234af => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:27.209603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d2d69-795f-438d-8f7e-733a9b45b819 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:27.210058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d2d69-795f-438d-8f7e-733a9b45b819 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:27.210486 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d2d69-795f-438d-8f7e-733a9b45b819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:27.229077 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-fd5d2d69-795f-438d-8f7e-733a9b45b819 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:27.230268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-fd5d2d69-795f-438d-8f7e-733a9b45b819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:27.247733 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:27.292984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1194/2383] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:33:27.301779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87b876-4cf9-406a-a1c5-e4104e1319aa None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:27.302071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87b876-4cf9-406a-a1c5-e4104e1319aa None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:27.302350 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87b876-4cf9-406a-a1c5-e4104e1319aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:27.320343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-7e87b876-4cf9-406a-a1c5-e4104e1319aa None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:27.321544 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7e87b876-4cf9-406a-a1c5-e4104e1319aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:27.329471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1190/2384] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:27.333583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d2d69-795f-438d-8f7e-733a9b45b819 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:27.343913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d2d69-795f-438d-8f7e-733a9b45b819 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:27.348420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb43309-e1b2-4759-84bb-a71c98a020b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:27.349061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb43309-e1b2-4759-84bb-a71c98a020b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:27.349341 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb43309-e1b2-4759-84bb-a71c98a020b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:27.349613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb43309-e1b2-4759-84bb-a71c98a020b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:27.350354 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb43309-e1b2-4759-84bb-a71c98a020b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:33:27.350697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb43309-e1b2-4759-84bb-a71c98a020b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:27.352331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb43309-e1b2-4759-84bb-a71c98a020b7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:27.368420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1195/2385] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:33:27.372671 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87b876-4cf9-406a-a1c5-e4104e1319aa None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:27.388134 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87b876-4cf9-406a-a1c5-e4104e1319aa None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:27.394100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c2d1d-a911-47d8-8cea-28263e001f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:27.395477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2d1d-a911-47d8-8cea-28263e001f28 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:27.395765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2d1d-a911-47d8-8cea-28263e001f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:27.396044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c2d1d-a911-47d8-8cea-28263e001f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:27.396752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c2d1d-a911-47d8-8cea-28263e001f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:27.400790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c2d1d-a911-47d8-8cea-28263e001f28 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:27.428255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1191/2386] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:33:27 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:33:27.432448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb43309-e1b2-4759-84bb-a71c98a020b7 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:27.442569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb43309-e1b2-4759-84bb-a71c98a020b7 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:27.498801 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61986f2-4166-4e17-bac3-2d8b432eb3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:27.499493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61986f2-4166-4e17-bac3-2d8b432eb3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:27.499785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61986f2-4166-4e17-bac3-2d8b432eb3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:27.500063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61986f2-4166-4e17-bac3-2d8b432eb3c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:27.500758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61986f2-4166-4e17-bac3-2d8b432eb3c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:27.502827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61986f2-4166-4e17-bac3-2d8b432eb3c9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:27.522199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1196/2387] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:33:27 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:33:27.526289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c2d1d-a911-47d8-8cea-28263e001f28 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:27.536918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c2d1d-a911-47d8-8cea-28263e001f28 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:27.541539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce903068-3d29-4752-9033-781ee26dfa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:27.542195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce903068-3d29-4752-9033-781ee26dfa58 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:27.542480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce903068-3d29-4752-9033-781ee26dfa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:27.542784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce903068-3d29-4752-9033-781ee26dfa58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:27.543557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce903068-3d29-4752-9033-781ee26dfa58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:27.545117 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce903068-3d29-4752-9033-781ee26dfa58 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:27.555531 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1192/2388] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:33:27.559584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61986f2-4166-4e17-bac3-2d8b432eb3c9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:27.569321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61986f2-4166-4e17-bac3-2d8b432eb3c9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:27.574452 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570683c-88d1-4af8-92e1-a8c3936cddef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:27.575240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570683c-88d1-4af8-92e1-a8c3936cddef admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:27.575524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570683c-88d1-4af8-92e1-a8c3936cddef admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:27.575800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570683c-88d1-4af8-92e1-a8c3936cddef admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:27.576519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570683c-88d1-4af8-92e1-a8c3936cddef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:27.578798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570683c-88d1-4af8-92e1-a8c3936cddef admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:27.589725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1197/2389] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:33:27.594445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce903068-3d29-4752-9033-781ee26dfa58 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:27.604945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce903068-3d29-4752-9033-781ee26dfa58 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:27.609577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c88c0f-30b3-47f6-a250-b695f1155c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:27.610447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c88c0f-30b3-47f6-a250-b695f1155c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:27.610769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c88c0f-30b3-47f6-a250-b695f1155c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:27.611046 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c88c0f-30b3-47f6-a250-b695f1155c41 admin admin] PATH_INFO: `/v3/projects/d5151421b3ee4477924cdab41025075b/users/5b82f76d685041d084dd5f332a5b1105/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:27.625322 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c88c0f-30b3-47f6-a250-b695f1155c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5151421b3ee4477924cdab41025075b, user_id=5b82f76d685041d084dd5f332a5b1105, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:27.628021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c88c0f-30b3-47f6-a250-b695f1155c41 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:27.641864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1193/2390] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:33:27 2022] PUT /identity/v3/projects/d5151421b3ee4477924cdab41025075b/users/5b82f76d685041d084dd5f332a5b1105/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:27.646865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8a373-fa76-41aa-bcb3-370abc087bc6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:27.647163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8a373-fa76-41aa-bcb3-370abc087bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:27.647448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8a373-fa76-41aa-bcb3-370abc087bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:27.667385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8fa8a373-fa76-41aa-bcb3-370abc087bc6 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:27.668601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8fa8a373-fa76-41aa-bcb3-370abc087bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:27.730693 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1198/2391] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:27.740485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c88c0f-30b3-47f6-a250-b695f1155c41 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:27.742210 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f1c88c0f-30b3-47f6-a250-b695f1155c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:27.764144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:27.801174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c88c0f-30b3-47f6-a250-b695f1155c41 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:27.804682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97d6ae-54c5-4747-8b55-990b5fae51a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:27.805317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97d6ae-54c5-4747-8b55-990b5fae51a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:27.805599 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97d6ae-54c5-4747-8b55-990b5fae51a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:27.805871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97d6ae-54c5-4747-8b55-990b5fae51a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:27.815267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97d6ae-54c5-4747-8b55-990b5fae51a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:27.816691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97d6ae-54c5-4747-8b55-990b5fae51a5 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:27.825965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1194/2392] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:33:28.471563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb8d4b-a192-4795-8a48-9243ab28372e None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:28.471857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb8d4b-a192-4795-8a48-9243ab28372e None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:28.472137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb8d4b-a192-4795-8a48-9243ab28372e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:28.501682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-01eb8d4b-a192-4795-8a48-9243ab28372e None None] MFA Rules not processed for user `5b82f76d685041d084dd5f332a5b1105`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:28.503171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-01eb8d4b-a192-4795-8a48-9243ab28372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:28.521836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:28.573366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1199/2393] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:28.584156 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97d6ae-54c5-4747-8b55-990b5fae51a5 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:28.585530 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ea97d6ae-54c5-4747-8b55-990b5fae51a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:28.655001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97d6ae-54c5-4747-8b55-990b5fae51a5 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:28.660503 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b823d4-a890-48fc-bb8a-56f31cf64d87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:28.661178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b823d4-a890-48fc-bb8a-56f31cf64d87 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:28.661459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b823d4-a890-48fc-bb8a-56f31cf64d87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:28.661734 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b823d4-a890-48fc-bb8a-56f31cf64d87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:28.672026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b823d4-a890-48fc-bb8a-56f31cf64d87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:28.673488 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b823d4-a890-48fc-bb8a-56f31cf64d87 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:28.683569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1195/2394] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:33:30.027954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a6c40-fbd8-495f-8d6d-77d08e0ee116 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:30.028513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a6c40-fbd8-495f-8d6d-77d08e0ee116 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:30.028513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a6c40-fbd8-495f-8d6d-77d08e0ee116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:30.046805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b21a6c40-fbd8-495f-8d6d-77d08e0ee116 None None] MFA Rules not processed for user `5b82f76d685041d084dd5f332a5b1105`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:30.047996 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b21a6c40-fbd8-495f-8d6d-77d08e0ee116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:30.055104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1200/2395] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:37.694789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b823d4-a890-48fc-bb8a-56f31cf64d87 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:37.696293 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-56b823d4-a890-48fc-bb8a-56f31cf64d87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:37.709030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b823d4-a890-48fc-bb8a-56f31cf64d87 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:37.713398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbeb4b6-ede9-42ff-999f-641dbc280ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:37.714109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbeb4b6-ede9-42ff-999f-641dbc280ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:37.714395 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbeb4b6-ede9-42ff-999f-641dbc280ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:37.714827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbeb4b6-ede9-42ff-999f-641dbc280ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:37.716603 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ecbeb4b6-ede9-42ff-999f-641dbc280ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:37.728715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbeb4b6-ede9-42ff-999f-641dbc280ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:37.730084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbeb4b6-ede9-42ff-999f-641dbc280ee4 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:37.739707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1196/2396] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:33:40.042105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=22 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:41.613564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a6c40-fbd8-495f-8d6d-77d08e0ee116 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:41.614939 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b21a6c40-fbd8-495f-8d6d-77d08e0ee116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:41.658854 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbeb4b6-ede9-42ff-999f-641dbc280ee4 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:41.660508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ecbeb4b6-ede9-42ff-999f-641dbc280ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:41.679069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbeb4b6-ede9-42ff-999f-641dbc280ee4 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:41.693575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ef304-1bac-48e1-9dcc-b51dc5a5c42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:41.694913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef304-1bac-48e1-9dcc-b51dc5a5c42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:41.695536 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef304-1bac-48e1-9dcc-b51dc5a5c42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:41.696034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef304-1bac-48e1-9dcc-b51dc5a5c42b admin admin] PATH_INFO: `/v3/users/193a35a91a47410f8b12cb2e84b33e1f` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:41.697993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ef304-1bac-48e1-9dcc-b51dc5a5c42b admin admin] RBAC: Authorizing `identity:delete_user(user_id=193a35a91a47410f8b12cb2e84b33e1f)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:41.700872 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ef304-1bac-48e1-9dcc-b51dc5a5c42b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:41.759796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a6c40-fbd8-495f-8d6d-77d08e0ee116 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:41.773165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8270d61-8ca6-459b-86cf-23b6b5c542e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:41.776093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8270d61-8ca6-459b-86cf-23b6b5c542e5 service placement] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:41.776699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8270d61-8ca6-459b-86cf-23b6b5c542e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:41.777366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8270d61-8ca6-459b-86cf-23b6b5c542e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:41.781120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a8270d61-8ca6-459b-86cf-23b6b5c542e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:41.786373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-096ef304-1bac-48e1-9dcc-b51dc5a5c42b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '193a35a91a47410f8b12cb2e84b33e1f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:41.800947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8270d61-8ca6-459b-86cf-23b6b5c542e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:41.803319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8270d61-8ca6-459b-86cf-23b6b5c542e5 service placement] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:41.803523 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-096ef304-1bac-48e1-9dcc-b51dc5a5c42b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '193a35a91a47410f8b12cb2e84b33e1f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:41.804823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-096ef304-1bac-48e1-9dcc-b51dc5a5c42b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '193a35a91a47410f8b12cb2e84b33e1f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:41.813834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-096ef304-1bac-48e1-9dcc-b51dc5a5c42b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '193a35a91a47410f8b12cb2e84b33e1f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:41.815490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-096ef304-1bac-48e1-9dcc-b51dc5a5c42b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '193a35a91a47410f8b12cb2e84b33e1f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:41.817107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1201/2397] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:33:41.837441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1197/2398] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:33:41 2022] DELETE /identity/v3/users/193a35a91a47410f8b12cb2e84b33e1f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:33:42.073521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8270d61-8ca6-459b-86cf-23b6b5c542e5 service placement] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:42.097364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8270d61-8ca6-459b-86cf-23b6b5c542e5 service placement] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:42.174424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:42.175365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:42.175648 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:42.175923 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] PATH_INFO: `/v3/projects/3e8f48e1939245eaa67bcd6a27bc7cda` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:42.177274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e8f48e1939245eaa67bcd6a27bc7cda)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:42.179246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:42.231832 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e8f48e1939245eaa67bcd6a27bc7cda', 'request_id': 'req-0ca7c184-58cd-431d-9895-35fbd22660ca'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:42.237212 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e8f48e1939245eaa67bcd6a27bc7cda', 'request_id': 'req-0ca7c184-58cd-431d-9895-35fbd22660ca'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:42.244937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e8f48e1939245eaa67bcd6a27bc7cda', 'request_id': 'req-0ca7c184-58cd-431d-9895-35fbd22660ca'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:42.256558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] The token cache is being invalidate because project 3e8f48e1939245eaa67bcd6a27bc7cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:33:42.256843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:42.257467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0ca7c184-58cd-431d-9895-35fbd22660ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:42.259121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1202/2399] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:33:42 2022] DELETE /identity/v3/projects/3e8f48e1939245eaa67bcd6a27bc7cda => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:42.265064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b86a-1227-406e-9feb-a4e859c09f6e None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:42.265353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b86a-1227-406e-9feb-a4e859c09f6e None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:42.265630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103b86a-1227-406e-9feb-a4e859c09f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:42.283086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-6103b86a-1227-406e-9feb-a4e859c09f6e None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:42.284597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6103b86a-1227-406e-9feb-a4e859c09f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:42.308062 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:42.353763 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1198/2400] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:42.361409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39268c-5572-42ad-8a13-739fe0285648 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:42.361689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39268c-5572-42ad-8a13-739fe0285648 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:42.361968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39268c-5572-42ad-8a13-739fe0285648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:42.385097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-dc39268c-5572-42ad-8a13-739fe0285648 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:42.386630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-dc39268c-5572-42ad-8a13-739fe0285648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:42.400439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1203/2401] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:42.404726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103b86a-1227-406e-9feb-a4e859c09f6e None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:42.414593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103b86a-1227-406e-9feb-a4e859c09f6e None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:42.419347 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4c765-7f5e-4e16-957a-7d5c0af25364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:42.420057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4c765-7f5e-4e16-957a-7d5c0af25364 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:42.420846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4c765-7f5e-4e16-957a-7d5c0af25364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:42.421119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4c765-7f5e-4e16-957a-7d5c0af25364 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:42.421873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4c765-7f5e-4e16-957a-7d5c0af25364 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:33:42.422171 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4c765-7f5e-4e16-957a-7d5c0af25364 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:42.423802 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4c765-7f5e-4e16-957a-7d5c0af25364 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:42.441302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1199/2402] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:33:42.442846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39268c-5572-42ad-8a13-739fe0285648 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:42.452599 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39268c-5572-42ad-8a13-739fe0285648 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:42.457537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29ca54-58c5-41df-a8ba-c4fb8aabd995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:42.458266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29ca54-58c5-41df-a8ba-c4fb8aabd995 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:42.458575 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29ca54-58c5-41df-a8ba-c4fb8aabd995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:42.458858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29ca54-58c5-41df-a8ba-c4fb8aabd995 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:42.459544 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29ca54-58c5-41df-a8ba-c4fb8aabd995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:42.461469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29ca54-58c5-41df-a8ba-c4fb8aabd995 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:42.471782 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:42.490912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1204/2403] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:33:42 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:33:42.495550 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4c765-7f5e-4e16-957a-7d5c0af25364 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:42.507282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4c765-7f5e-4e16-957a-7d5c0af25364 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:42.566712 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecff525-3abd-43d8-8b3e-54b6d2020ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:42.567381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecff525-3abd-43d8-8b3e-54b6d2020ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:42.567666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecff525-3abd-43d8-8b3e-54b6d2020ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:42.567938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecff525-3abd-43d8-8b3e-54b6d2020ed8 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:42.568619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecff525-3abd-43d8-8b3e-54b6d2020ed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:42.570572 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecff525-3abd-43d8-8b3e-54b6d2020ed8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:42.592699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1200/2404] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:33:42 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:33:42.596881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29ca54-58c5-41df-a8ba-c4fb8aabd995 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:42.608928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29ca54-58c5-41df-a8ba-c4fb8aabd995 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:42.614041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771caf0f-7837-4d3c-bc34-de0d541d0c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:42.614876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771caf0f-7837-4d3c-bc34-de0d541d0c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:42.615010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771caf0f-7837-4d3c-bc34-de0d541d0c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:42.615293 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771caf0f-7837-4d3c-bc34-de0d541d0c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:42.616048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771caf0f-7837-4d3c-bc34-de0d541d0c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:42.618253 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771caf0f-7837-4d3c-bc34-de0d541d0c4d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:42.630777 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1205/2405] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:33:42.634863 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecff525-3abd-43d8-8b3e-54b6d2020ed8 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:42.645239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecff525-3abd-43d8-8b3e-54b6d2020ed8 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:42.650638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fa8b8-f3a1-4191-b62d-525acfc8a317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:42.651430 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa8b8-f3a1-4191-b62d-525acfc8a317 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:42.651841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa8b8-f3a1-4191-b62d-525acfc8a317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:42.652194 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fa8b8-f3a1-4191-b62d-525acfc8a317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:42.654605 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fa8b8-f3a1-4191-b62d-525acfc8a317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:42.656508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fa8b8-f3a1-4191-b62d-525acfc8a317 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:42.668623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1201/2406] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:33:42.673559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771caf0f-7837-4d3c-bc34-de0d541d0c4d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:42.685143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771caf0f-7837-4d3c-bc34-de0d541d0c4d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:42.692000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bf43a-a142-4ec7-9fc9-155a6d1e53f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:42.692903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bf43a-a142-4ec7-9fc9-155a6d1e53f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:42.693187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bf43a-a142-4ec7-9fc9-155a6d1e53f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:42.693645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919bf43a-a142-4ec7-9fc9-155a6d1e53f3 admin admin] PATH_INFO: `/v3/projects/5f3e45fa4b0646238fd986f5cbb2e423/users/cb44e47aa5194b4fa4b28c1f92097d01/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:42.709849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bf43a-a142-4ec7-9fc9-155a6d1e53f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3e45fa4b0646238fd986f5cbb2e423, user_id=cb44e47aa5194b4fa4b28c1f92097d01, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:42.712785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919bf43a-a142-4ec7-9fc9-155a6d1e53f3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:42.725350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1206/2407] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:33:42 2022] PUT /identity/v3/projects/5f3e45fa4b0646238fd986f5cbb2e423/users/cb44e47aa5194b4fa4b28c1f92097d01/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:33:42.736537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c5ca2-58d6-476a-a794-36b00646be98 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:42.736823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c5ca2-58d6-476a-a794-36b00646be98 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:42.737102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c5ca2-58d6-476a-a794-36b00646be98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:42.765360 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-032c5ca2-58d6-476a-a794-36b00646be98 None None] MFA Rules not processed for user `cb44e47aa5194b4fa4b28c1f92097d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:42.766630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-032c5ca2-58d6-476a-a794-36b00646be98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:42.835016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1202/2408] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:42.846915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bf43a-a142-4ec7-9fc9-155a6d1e53f3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:42.848635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-919bf43a-a142-4ec7-9fc9-155a6d1e53f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:42.910475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919bf43a-a142-4ec7-9fc9-155a6d1e53f3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:42.916765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f76b4-e4f1-4647-823d-b6d1407c2df2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:42.917174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f76b4-e4f1-4647-823d-b6d1407c2df2 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:42.917478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f76b4-e4f1-4647-823d-b6d1407c2df2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:42.917758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f76b4-e4f1-4647-823d-b6d1407c2df2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:42.927893 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f76b4-e4f1-4647-823d-b6d1407c2df2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:42.929334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f76b4-e4f1-4647-823d-b6d1407c2df2 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:42.938469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1207/2409] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:33:45.452491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c5ca2-58d6-476a-a794-36b00646be98 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:45.453961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-032c5ca2-58d6-476a-a794-36b00646be98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:45.475920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:45.513424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c5ca2-58d6-476a-a794-36b00646be98 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:45.516993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c7d3b-9001-4279-b6cc-363c0e62a40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:45.517637 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c7d3b-9001-4279-b6cc-363c0e62a40e service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:45.517916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c7d3b-9001-4279-b6cc-363c0e62a40e service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:45.518187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c7d3b-9001-4279-b6cc-363c0e62a40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:45.520003 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-228c7d3b-9001-4279-b6cc-363c0e62a40e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:45.579782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c7d3b-9001-4279-b6cc-363c0e62a40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:45.581295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c7d3b-9001-4279-b6cc-363c0e62a40e service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:45.590968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1203/2410] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:33:50.856528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f76b4-e4f1-4647-823d-b6d1407c2df2 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:50.859205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8a8f76b4-e4f1-4647-823d-b6d1407c2df2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:50.874487 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f76b4-e4f1-4647-823d-b6d1407c2df2 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:50.877665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30533164-ef1d-47c6-a992-9673b49f0207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:50.878310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30533164-ef1d-47c6-a992-9673b49f0207 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:50.878620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30533164-ef1d-47c6-a992-9673b49f0207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:50.878896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30533164-ef1d-47c6-a992-9673b49f0207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:50.880773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-30533164-ef1d-47c6-a992-9673b49f0207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:50.950243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30533164-ef1d-47c6-a992-9673b49f0207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:50.951767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30533164-ef1d-47c6-a992-9673b49f0207 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:50.962193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1208/2411] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:33:51.818559 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c7d3b-9001-4279-b6cc-363c0e62a40e service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:51.834323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c7d3b-9001-4279-b6cc-363c0e62a40e service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:51.840127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:51.841022 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:51.841302 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:51.842187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] PATH_INFO: `/v3/users/cb44e47aa5194b4fa4b28c1f92097d01` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:51.855918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb44e47aa5194b4fa4b28c1f92097d01)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:51.868163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:51.935290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb44e47aa5194b4fa4b28c1f92097d01'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:51.950693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb44e47aa5194b4fa4b28c1f92097d01'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:51.951849 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb44e47aa5194b4fa4b28c1f92097d01'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:51.960670 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb44e47aa5194b4fa4b28c1f92097d01'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:51.964325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb44e47aa5194b4fa4b28c1f92097d01'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:52.006595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1204/2412] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:33:51 2022] DELETE /identity/v3/users/cb44e47aa5194b4fa4b28c1f92097d01 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:52.011592 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90581ea-c999-4e68-8534-0d043474bef7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:52.012100 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90581ea-c999-4e68-8534-0d043474bef7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:52.012188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90581ea-c999-4e68-8534-0d043474bef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:52.036595 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b90581ea-c999-4e68-8534-0d043474bef7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:52.038230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b90581ea-c999-4e68-8534-0d043474bef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:52.110862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:52.128533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1209/2413] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:52.140152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:52.141869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:52.213496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:52.220394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ec863-85a7-4f17-a9e9-3c2ea687041d admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:52.226235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fe219-400d-4efb-b622-423062b6047e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:52.227081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fe219-400d-4efb-b622-423062b6047e service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:52.227813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fe219-400d-4efb-b622-423062b6047e service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:52.228651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fe219-400d-4efb-b622-423062b6047e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:52.240328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fe219-400d-4efb-b622-423062b6047e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:52.241818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fe219-400d-4efb-b622-423062b6047e service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:52.252621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1205/2414] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:33:52.593708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90581ea-c999-4e68-8534-0d043474bef7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:52.595280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b90581ea-c999-4e68-8534-0d043474bef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:52.618916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90581ea-c999-4e68-8534-0d043474bef7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:52.635552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:52.636667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:52.637065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:52.637442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] PATH_INFO: `/v3/projects/5f3e45fa4b0646238fd986f5cbb2e423` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:52.648948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f3e45fa4b0646238fd986f5cbb2e423)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:52.649788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:52.728708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f3e45fa4b0646238fd986f5cbb2e423', 'request_id': 'req-173144b2-6b4b-410d-aa87-50815349522d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:52.735317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f3e45fa4b0646238fd986f5cbb2e423', 'request_id': 'req-173144b2-6b4b-410d-aa87-50815349522d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:52.745929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f3e45fa4b0646238fd986f5cbb2e423', 'request_id': 'req-173144b2-6b4b-410d-aa87-50815349522d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:52.770191 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] The token cache is being invalidate because project 5f3e45fa4b0646238fd986f5cbb2e423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:33:52.770827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:52.773569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-173144b2-6b4b-410d-aa87-50815349522d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:52.779580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1210/2415] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:33:52 2022] DELETE /identity/v3/projects/5f3e45fa4b0646238fd986f5cbb2e423 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:52.781317 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4558a6-3d48-4552-8f21-1e23201e9175 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:52.781618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4558a6-3d48-4552-8f21-1e23201e9175 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:52.781896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4558a6-3d48-4552-8f21-1e23201e9175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:52.800790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-de4558a6-3d48-4552-8f21-1e23201e9175 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:52.802036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-de4558a6-3d48-4552-8f21-1e23201e9175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:52.903459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1206/2416] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:33:52.906746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d98210-6df1-424f-8e3c-d0de893245f8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:52.907075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d98210-6df1-424f-8e3c-d0de893245f8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:52.907392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d98210-6df1-424f-8e3c-d0de893245f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:52.935078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-32d98210-6df1-424f-8e3c-d0de893245f8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:52.937363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-32d98210-6df1-424f-8e3c-d0de893245f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:52.950051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1211/2417] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:52.954836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4558a6-3d48-4552-8f21-1e23201e9175 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:52.968478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4558a6-3d48-4552-8f21-1e23201e9175 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:52.973794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed77908-e6c3-4d9a-8c15-b5a068c0fd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:52.974567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed77908-e6c3-4d9a-8c15-b5a068c0fd54 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:52.974913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed77908-e6c3-4d9a-8c15-b5a068c0fd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:52.975314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed77908-e6c3-4d9a-8c15-b5a068c0fd54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:52.975938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed77908-e6c3-4d9a-8c15-b5a068c0fd54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:33:52.976455 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed77908-e6c3-4d9a-8c15-b5a068c0fd54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:52.978193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed77908-e6c3-4d9a-8c15-b5a068c0fd54 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:52.996190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d98210-6df1-424f-8e3c-d0de893245f8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:53.003081 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1207/2418] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:33:53.006441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d98210-6df1-424f-8e3c-d0de893245f8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:53.014246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffec4e-5026-4c87-85b8-9f9bb8927632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:53.015846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffec4e-5026-4c87-85b8-9f9bb8927632 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:53.016390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffec4e-5026-4c87-85b8-9f9bb8927632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:53.017091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffec4e-5026-4c87-85b8-9f9bb8927632 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:53.017821 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffec4e-5026-4c87-85b8-9f9bb8927632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:53.020357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffec4e-5026-4c87-85b8-9f9bb8927632 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:53.053276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1212/2419] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:33:52 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:33:53.054892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed77908-e6c3-4d9a-8c15-b5a068c0fd54 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:53.068312 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed77908-e6c3-4d9a-8c15-b5a068c0fd54 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:53.145039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b2bb3-3ee5-437a-9f77-d3542c0c7237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:53.145039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2bb3-3ee5-437a-9f77-d3542c0c7237 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:53.145326 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2bb3-3ee5-437a-9f77-d3542c0c7237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:53.145617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b2bb3-3ee5-437a-9f77-d3542c0c7237 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:53.148096 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b2bb3-3ee5-437a-9f77-d3542c0c7237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:53.149082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b2bb3-3ee5-437a-9f77-d3542c0c7237 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:53.181239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1208/2420] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:33:53 2022] POST /identity/v3/users => generated 303 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:33:53.186162 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffec4e-5026-4c87-85b8-9f9bb8927632 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:53.202238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffec4e-5026-4c87-85b8-9f9bb8927632 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:53.216339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a46be6-d72e-461b-a128-2701234747f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:53.217206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a46be6-d72e-461b-a128-2701234747f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:53.217492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a46be6-d72e-461b-a128-2701234747f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:53.217765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a46be6-d72e-461b-a128-2701234747f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:53.218498 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a46be6-d72e-461b-a128-2701234747f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:53.220166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a46be6-d72e-461b-a128-2701234747f3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:53.233388 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1213/2421] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:33:53.238378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b2bb3-3ee5-437a-9f77-d3542c0c7237 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:53.251641 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b2bb3-3ee5-437a-9f77-d3542c0c7237 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:53.259483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd02afb-19a1-4da6-8ab5-3a4c8c8fa586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:53.260454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd02afb-19a1-4da6-8ab5-3a4c8c8fa586 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:53.261033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd02afb-19a1-4da6-8ab5-3a4c8c8fa586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:53.261373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd02afb-19a1-4da6-8ab5-3a4c8c8fa586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:53.262126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd02afb-19a1-4da6-8ab5-3a4c8c8fa586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:53.263886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd02afb-19a1-4da6-8ab5-3a4c8c8fa586 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:53.275639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1209/2422] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:33:53.279849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a46be6-d72e-461b-a128-2701234747f3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:53.290677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a46be6-d72e-461b-a128-2701234747f3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:53.296323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc5cd5-a55b-432c-a2bd-29ff43d86809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:53.297213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc5cd5-a55b-432c-a2bd-29ff43d86809 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:53.297499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc5cd5-a55b-432c-a2bd-29ff43d86809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:53.297778 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc5cd5-a55b-432c-a2bd-29ff43d86809 admin admin] PATH_INFO: `/v3/projects/ca1ecc6439664854adbe99b38a02babe/users/289d361fd0d94ad8b31533f4434130e3/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:53.314856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc5cd5-a55b-432c-a2bd-29ff43d86809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca1ecc6439664854adbe99b38a02babe, user_id=289d361fd0d94ad8b31533f4434130e3, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:53.317583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc5cd5-a55b-432c-a2bd-29ff43d86809 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:53.332512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1214/2423] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:33:53 2022] PUT /identity/v3/projects/ca1ecc6439664854adbe99b38a02babe/users/289d361fd0d94ad8b31533f4434130e3/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:53.349956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd6673-05a8-4b0a-841c-e9d7470a8393 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:53.350602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd6673-05a8-4b0a-841c-e9d7470a8393 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:53.350889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd6673-05a8-4b0a-841c-e9d7470a8393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:53.380011 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-95cd6673-05a8-4b0a-841c-e9d7470a8393 None None] MFA Rules not processed for user `289d361fd0d94ad8b31533f4434130e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:53.381242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-95cd6673-05a8-4b0a-841c-e9d7470a8393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:53.427349 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:53.453726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1210/2424] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:53 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:53.458281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc5cd5-a55b-432c-a2bd-29ff43d86809 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:53.526846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc5cd5-a55b-432c-a2bd-29ff43d86809 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:53.534005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a87ca-2e53-471a-ad11-223d7f6f2bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:53.534746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a87ca-2e53-471a-ad11-223d7f6f2bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:53.535533 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a87ca-2e53-471a-ad11-223d7f6f2bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:53.535818 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a87ca-2e53-471a-ad11-223d7f6f2bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:53.536918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a87ca-2e53-471a-ad11-223d7f6f2bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:53.539928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a87ca-2e53-471a-ad11-223d7f6f2bd5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:53.570304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:53.589152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd6673-05a8-4b0a-841c-e9d7470a8393 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:53.592916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1215/2425] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:33:53 2022] POST /identity/v3/projects => generated 405 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:33:53.605709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd6673-05a8-4b0a-841c-e9d7470a8393 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:53.707592 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa084e-5689-429f-b8c2-b1820a0de459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:53.708881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa084e-5689-429f-b8c2-b1820a0de459 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:53.709044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa084e-5689-429f-b8c2-b1820a0de459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:53.709443 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa084e-5689-429f-b8c2-b1820a0de459 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:53.710381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa084e-5689-429f-b8c2-b1820a0de459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:53.713074 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa084e-5689-429f-b8c2-b1820a0de459 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:53.742309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1211/2426] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:33:53 2022] POST /identity/v3/users => generated 309 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:33:53.745849 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a87ca-2e53-471a-ad11-223d7f6f2bd5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:53.766113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a87ca-2e53-471a-ad11-223d7f6f2bd5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:53.784782 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee585c7-356d-48e2-b9c8-84f13c65ba42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:53.785517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee585c7-356d-48e2-b9c8-84f13c65ba42 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:53.785798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee585c7-356d-48e2-b9c8-84f13c65ba42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:53.786240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee585c7-356d-48e2-b9c8-84f13c65ba42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:53.787361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee585c7-356d-48e2-b9c8-84f13c65ba42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:53.790229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee585c7-356d-48e2-b9c8-84f13c65ba42 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:53.809149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa084e-5689-429f-b8c2-b1820a0de459 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:53.816272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1216/2427] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:33:53.822983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa084e-5689-429f-b8c2-b1820a0de459 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:53.828717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3591e09-5ae2-4f4c-9821-549f531d0259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:53.829834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3591e09-5ae2-4f4c-9821-549f531d0259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:53.830437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3591e09-5ae2-4f4c-9821-549f531d0259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:53.830437 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3591e09-5ae2-4f4c-9821-549f531d0259 admin admin] PATH_INFO: `/v3/projects/9330294a43c6414ca714db9dd8740b13/users/05b50333de8342789186a165e41fffcf/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:53.850957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3591e09-5ae2-4f4c-9821-549f531d0259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9330294a43c6414ca714db9dd8740b13, user_id=05b50333de8342789186a165e41fffcf, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:53.855028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3591e09-5ae2-4f4c-9821-549f531d0259 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:53.893405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee585c7-356d-48e2-b9c8-84f13c65ba42 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:53.893771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1212/2428] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:33:53 2022] PUT /identity/v3/projects/9330294a43c6414ca714db9dd8740b13/users/05b50333de8342789186a165e41fffcf/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:53.901376 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee585c7-356d-48e2-b9c8-84f13c65ba42 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:53.937275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3591e09-5ae2-4f4c-9821-549f531d0259 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:53.939350 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b3591e09-5ae2-4f4c-9821-549f531d0259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:53.991375 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fc523-d099-4dee-bb11-5ff68b6ebc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:53.992438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fc523-d099-4dee-bb11-5ff68b6ebc65 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:53.993030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fc523-d099-4dee-bb11-5ff68b6ebc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:53.993483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fc523-d099-4dee-bb11-5ff68b6ebc65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:53.994472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fc523-d099-4dee-bb11-5ff68b6ebc65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:53.996133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fc523-d099-4dee-bb11-5ff68b6ebc65 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:54.008366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1217/2429] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:33:54.012649 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fc523-d099-4dee-bb11-5ff68b6ebc65 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:54.014256 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3591e09-5ae2-4f4c-9821-549f531d0259 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:54.018733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df256e-6d9f-416c-b4f3-1ff5dc055cdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:54.019914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df256e-6d9f-416c-b4f3-1ff5dc055cdc service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:54.021128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df256e-6d9f-416c-b4f3-1ff5dc055cdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:54.021558 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df256e-6d9f-416c-b4f3-1ff5dc055cdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:54.024170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-73df256e-6d9f-416c-b4f3-1ff5dc055cdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:54.024634 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fc523-d099-4dee-bb11-5ff68b6ebc65 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:54.034968 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2223dccf-42c5-4d64-bd67-4b00bf388012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:54.035879 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223dccf-42c5-4d64-bd67-4b00bf388012 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:54.036166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223dccf-42c5-4d64-bd67-4b00bf388012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:54.036446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2223dccf-42c5-4d64-bd67-4b00bf388012 admin admin] PATH_INFO: `/v3/projects/9330294a43c6414ca714db9dd8740b13/users/05b50333de8342789186a165e41fffcf/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:54.038965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223dccf-42c5-4d64-bd67-4b00bf388012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9330294a43c6414ca714db9dd8740b13, user_id=05b50333de8342789186a165e41fffcf, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:54.043538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2223dccf-42c5-4d64-bd67-4b00bf388012 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:54.048305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df256e-6d9f-416c-b4f3-1ff5dc055cdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:54.050960 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df256e-6d9f-416c-b4f3-1ff5dc055cdc service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:54.059898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1218/2430] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:33:54 2022] PUT /identity/v3/projects/9330294a43c6414ca714db9dd8740b13/users/05b50333de8342789186a165e41fffcf/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:54.077287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1213/2431] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:33:54.081183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addbe9c-31f3-489d-8234-1dab443a7a01 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:54.081555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addbe9c-31f3-489d-8234-1dab443a7a01 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:54.081945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addbe9c-31f3-489d-8234-1dab443a7a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:54.149502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9addbe9c-31f3-489d-8234-1dab443a7a01 None None] MFA Rules not processed for user `05b50333de8342789186a165e41fffcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:54.151171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9addbe9c-31f3-489d-8234-1dab443a7a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:54.307802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1219/2432] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:54 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:54.315495 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df256e-6d9f-416c-b4f3-1ff5dc055cdc service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:54.317222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-73df256e-6d9f-416c-b4f3-1ff5dc055cdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:54.430386 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df256e-6d9f-416c-b4f3-1ff5dc055cdc service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:54.440231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:54.518478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e5acf-0024-443c-b86f-ce641092fdc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:54.519343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e5acf-0024-443c-b86f-ce641092fdc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:54.519626 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e5acf-0024-443c-b86f-ce641092fdc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:54.519897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e5acf-0024-443c-b86f-ce641092fdc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:54.540946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e5acf-0024-443c-b86f-ce641092fdc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:54.543893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e5acf-0024-443c-b86f-ce641092fdc2 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:54.560491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1214/2433] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:54 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:33:54.774274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addbe9c-31f3-489d-8234-1dab443a7a01 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:54.846883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:54.856414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9addbe9c-31f3-489d-8234-1dab443a7a01 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:54.863467 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:54.864460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:54.864866 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:54.865207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] PATH_INFO: `/v3/users/289d361fd0d94ad8b31533f4434130e3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:54.866461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=289d361fd0d94ad8b31533f4434130e3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:54.868603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:54.943606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '289d361fd0d94ad8b31533f4434130e3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:54.966338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '289d361fd0d94ad8b31533f4434130e3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:54.967377 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '289d361fd0d94ad8b31533f4434130e3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:54.983862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '289d361fd0d94ad8b31533f4434130e3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:54.984612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '289d361fd0d94ad8b31533f4434130e3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:55.002766 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1220/2434] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:33:54 2022] DELETE /identity/v3/users/289d361fd0d94ad8b31533f4434130e3 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:55.006992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501327b-4c38-4c47-87d5-710436fb2f48 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:55.008593 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501327b-4c38-4c47-87d5-710436fb2f48 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:55.010934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501327b-4c38-4c47-87d5-710436fb2f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:55.034868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-a501327b-4c38-4c47-87d5-710436fb2f48 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:55.037477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a501327b-4c38-4c47-87d5-710436fb2f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:55.139062 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1215/2435] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:33:55.150316 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:55.158179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:55.256271 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d810aa-9c83-4099-a6f1-808b82a8bbaf admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:55.261794 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685613fe-370f-4e6c-b65c-07ac69aa1ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:55.269373 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685613fe-370f-4e6c-b65c-07ac69aa1ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:55.269675 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685613fe-370f-4e6c-b65c-07ac69aa1ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:55.269974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685613fe-370f-4e6c-b65c-07ac69aa1ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:55.284055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685613fe-370f-4e6c-b65c-07ac69aa1ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:55.285535 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685613fe-370f-4e6c-b65c-07ac69aa1ab4 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:55.298577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1221/2436] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:33:56.247282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501327b-4c38-4c47-87d5-710436fb2f48 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:56.252825 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-a501327b-4c38-4c47-87d5-710436fb2f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:56.280581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501327b-4c38-4c47-87d5-710436fb2f48 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:56.290614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:56.291796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:56.292349 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:56.292813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] PATH_INFO: `/v3/projects/ca1ecc6439664854adbe99b38a02babe` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:56.295129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca1ecc6439664854adbe99b38a02babe)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:56.297799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:56.413194 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca1ecc6439664854adbe99b38a02babe', 'request_id': 'req-4a4360cf-2b33-4571-a83a-37ed7afb73a5'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:56.430123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca1ecc6439664854adbe99b38a02babe', 'request_id': 'req-4a4360cf-2b33-4571-a83a-37ed7afb73a5'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:56.442493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca1ecc6439664854adbe99b38a02babe', 'request_id': 'req-4a4360cf-2b33-4571-a83a-37ed7afb73a5'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:56.458512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] The token cache is being invalidate because project ca1ecc6439664854adbe99b38a02babe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:33:56.459106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:56.459946 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:56.463135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1216/2437] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:33:56 2022] DELETE /identity/v3/projects/ca1ecc6439664854adbe99b38a02babe => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:56.469095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685613fe-370f-4e6c-b65c-07ac69aa1ab4 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:56.484098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685613fe-370f-4e6c-b65c-07ac69aa1ab4 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:56.580119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a7f3c-2470-44ba-8a78-e40c14aa5d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:56.581174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a7f3c-2470-44ba-8a78-e40c14aa5d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:56.581514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a7f3c-2470-44ba-8a78-e40c14aa5d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:56.581792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a7f3c-2470-44ba-8a78-e40c14aa5d1c admin admin] PATH_INFO: `/v3/users/05b50333de8342789186a165e41fffcf` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:56.583255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a7f3c-2470-44ba-8a78-e40c14aa5d1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b50333de8342789186a165e41fffcf)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:56.585410 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a7f3c-2470-44ba-8a78-e40c14aa5d1c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:56.611684 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:56.642938 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-770a7f3c-2470-44ba-8a78-e40c14aa5d1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b50333de8342789186a165e41fffcf'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:56.657876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-770a7f3c-2470-44ba-8a78-e40c14aa5d1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b50333de8342789186a165e41fffcf'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:56.658805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-770a7f3c-2470-44ba-8a78-e40c14aa5d1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b50333de8342789186a165e41fffcf'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:56.667259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-770a7f3c-2470-44ba-8a78-e40c14aa5d1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b50333de8342789186a165e41fffcf'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:56.668698 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-770a7f3c-2470-44ba-8a78-e40c14aa5d1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b50333de8342789186a165e41fffcf'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:56.688706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1222/2438] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:33:56 2022] DELETE /identity/v3/users/05b50333de8342789186a165e41fffcf => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:33:57.000365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:57.002468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:57.018304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4360cf-2b33-4571-a83a-37ed7afb73a5 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:57.058318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:57.089981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:57.090823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:57.091108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:57.091921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] PATH_INFO: `/v3/projects/9330294a43c6414ca714db9dd8740b13` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:57.093151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9330294a43c6414ca714db9dd8740b13)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:57.095187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:57.149994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9330294a43c6414ca714db9dd8740b13', 'request_id': 'req-459eb62b-5ea5-4008-8ef8-b6ffad3da707'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:57.156901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9330294a43c6414ca714db9dd8740b13', 'request_id': 'req-459eb62b-5ea5-4008-8ef8-b6ffad3da707'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:57.172337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9330294a43c6414ca714db9dd8740b13', 'request_id': 'req-459eb62b-5ea5-4008-8ef8-b6ffad3da707'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:57.187101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] The token cache is being invalidate because project 9330294a43c6414ca714db9dd8740b13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:33:57.187393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:57.188056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-459eb62b-5ea5-4008-8ef8-b6ffad3da707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:33:57.189631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1217/2439] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:33:56 2022] DELETE /identity/v3/projects/9330294a43c6414ca714db9dd8740b13 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:57.194860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850dbbd-e293-43a6-94ad-664bf9c9e423 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:57.195242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850dbbd-e293-43a6-94ad-664bf9c9e423 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:57.195519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850dbbd-e293-43a6-94ad-664bf9c9e423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:57.215166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c850dbbd-e293-43a6-94ad-664bf9c9e423 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:57.216522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c850dbbd-e293-43a6-94ad-664bf9c9e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:57.280731 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1223/2440] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:33:57.288770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4381ca-5885-43b2-b4ab-43725e8ee30c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:57.289818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4381ca-5885-43b2-b4ab-43725e8ee30c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:57.290093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4381ca-5885-43b2-b4ab-43725e8ee30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:57.308661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-cd4381ca-5885-43b2-b4ab-43725e8ee30c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:57.309883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-cd4381ca-5885-43b2-b4ab-43725e8ee30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:57.318633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1218/2441] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:57.323896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850dbbd-e293-43a6-94ad-664bf9c9e423 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:57.336988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850dbbd-e293-43a6-94ad-664bf9c9e423 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:57.342371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18805d7f-5898-4c36-9684-2e7731d4adbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:57.343016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18805d7f-5898-4c36-9684-2e7731d4adbb admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:57.343299 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18805d7f-5898-4c36-9684-2e7731d4adbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:57.343571 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18805d7f-5898-4c36-9684-2e7731d4adbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:57.344621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18805d7f-5898-4c36-9684-2e7731d4adbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:33:57.345183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18805d7f-5898-4c36-9684-2e7731d4adbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:57.347835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18805d7f-5898-4c36-9684-2e7731d4adbb admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:57.366653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1224/2442] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:33:57.368213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4381ca-5885-43b2-b4ab-43725e8ee30c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:57.378618 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4381ca-5885-43b2-b4ab-43725e8ee30c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:57.383580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37e83a-e99f-4859-bfe6-e2f2530d7996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:57.384253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37e83a-e99f-4859-bfe6-e2f2530d7996 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:57.384533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37e83a-e99f-4859-bfe6-e2f2530d7996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:57.384838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37e83a-e99f-4859-bfe6-e2f2530d7996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:57.385789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37e83a-e99f-4859-bfe6-e2f2530d7996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:57.387796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37e83a-e99f-4859-bfe6-e2f2530d7996 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:57.414941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1219/2443] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:33:57 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:33:57.419279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18805d7f-5898-4c36-9684-2e7731d4adbb admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:57.430176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18805d7f-5898-4c36-9684-2e7731d4adbb admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:57.574133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5987d-81d6-4030-8540-b3dd9a29d351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:57.575665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5987d-81d6-4030-8540-b3dd9a29d351 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:57.576188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5987d-81d6-4030-8540-b3dd9a29d351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:57.576705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5987d-81d6-4030-8540-b3dd9a29d351 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:57.577629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5987d-81d6-4030-8540-b3dd9a29d351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:57.588974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5987d-81d6-4030-8540-b3dd9a29d351 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:57.622113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1225/2444] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:33:57 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:33:57.627412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37e83a-e99f-4859-bfe6-e2f2530d7996 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:57.643140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37e83a-e99f-4859-bfe6-e2f2530d7996 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:57.657447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1959320-93af-4119-9894-fbd3a5902e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:57.658695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1959320-93af-4119-9894-fbd3a5902e09 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:57.659185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1959320-93af-4119-9894-fbd3a5902e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:57.659664 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1959320-93af-4119-9894-fbd3a5902e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:57.660620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1959320-93af-4119-9894-fbd3a5902e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:57.663384 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1959320-93af-4119-9894-fbd3a5902e09 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:57.695854 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1220/2445] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:33:57.701523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5987d-81d6-4030-8540-b3dd9a29d351 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:57.718121 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5987d-81d6-4030-8540-b3dd9a29d351 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:57.730513 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30efd37-7c6e-4ba9-822a-2994dffa4cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:57.732868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30efd37-7c6e-4ba9-822a-2994dffa4cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:57.733327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30efd37-7c6e-4ba9-822a-2994dffa4cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:57.734003 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30efd37-7c6e-4ba9-822a-2994dffa4cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:57.736378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30efd37-7c6e-4ba9-822a-2994dffa4cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:57.738962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30efd37-7c6e-4ba9-822a-2994dffa4cb0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:57.766557 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1226/2446] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:33:57.771059 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1959320-93af-4119-9894-fbd3a5902e09 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:57.784733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1959320-93af-4119-9894-fbd3a5902e09 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:57.800312 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e01a3d-f6e7-4424-80b5-2ef90d772c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:57.801567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e01a3d-f6e7-4424-80b5-2ef90d772c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:57.801848 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e01a3d-f6e7-4424-80b5-2ef90d772c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:57.802117 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e01a3d-f6e7-4424-80b5-2ef90d772c8f admin admin] PATH_INFO: `/v3/projects/0b3962c8fc204046af26c5f91d3ded13/users/c182527fdae74c84b4c4db6f40c4904b/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:57.837538 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e01a3d-f6e7-4424-80b5-2ef90d772c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b3962c8fc204046af26c5f91d3ded13, user_id=c182527fdae74c84b4c4db6f40c4904b, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:57.843547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e01a3d-f6e7-4424-80b5-2ef90d772c8f admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:57.868835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1221/2447] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:33:57 2022] PUT /identity/v3/projects/0b3962c8fc204046af26c5f91d3ded13/users/c182527fdae74c84b4c4db6f40c4904b/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:57.884124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e4a47-3767-4adb-b6be-0daff9330ddd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:57.884564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e4a47-3767-4adb-b6be-0daff9330ddd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:57.884940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e4a47-3767-4adb-b6be-0daff9330ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:57.899460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:57.951698 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b33e4a47-3767-4adb-b6be-0daff9330ddd None None] MFA Rules not processed for user `c182527fdae74c84b4c4db6f40c4904b`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:57.954115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b33e4a47-3767-4adb-b6be-0daff9330ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:58.090775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1227/2448] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:33:58.095551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e01a3d-f6e7-4424-80b5-2ef90d772c8f admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:58.240145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e01a3d-f6e7-4424-80b5-2ef90d772c8f admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:58.264499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989938fe-2a6b-476b-9d84-c2a9082c0631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:58.265456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989938fe-2a6b-476b-9d84-c2a9082c0631 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:58.265933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989938fe-2a6b-476b-9d84-c2a9082c0631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:58.266604 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989938fe-2a6b-476b-9d84-c2a9082c0631 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:58.267551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989938fe-2a6b-476b-9d84-c2a9082c0631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:58.270010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989938fe-2a6b-476b-9d84-c2a9082c0631 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:58.275077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:58.308324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1222/2449] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:33:58 2022] POST /identity/v3/projects => generated 379 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:33:58.313130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e4a47-3767-4adb-b6be-0daff9330ddd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:58.324606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e4a47-3767-4adb-b6be-0daff9330ddd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:58.396633 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d7bc2-3dce-4157-b03a-8c10c1e51737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:58.396871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d7bc2-3dce-4157-b03a-8c10c1e51737 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:58.397520 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d7bc2-3dce-4157-b03a-8c10c1e51737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:58.398031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d7bc2-3dce-4157-b03a-8c10c1e51737 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:58.398714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d7bc2-3dce-4157-b03a-8c10c1e51737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:58.401580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d7bc2-3dce-4157-b03a-8c10c1e51737 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:58.427902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1228/2450] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:33:58 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:33:58.433254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989938fe-2a6b-476b-9d84-c2a9082c0631 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:58.445796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989938fe-2a6b-476b-9d84-c2a9082c0631 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:58.452233 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f873d37-8d48-4332-90f3-30b8e2bf11ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:58.453512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f873d37-8d48-4332-90f3-30b8e2bf11ce admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:58.453805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f873d37-8d48-4332-90f3-30b8e2bf11ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:58.454092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f873d37-8d48-4332-90f3-30b8e2bf11ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:58.454877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f873d37-8d48-4332-90f3-30b8e2bf11ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:58.456513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f873d37-8d48-4332-90f3-30b8e2bf11ce admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:58.472049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1223/2451] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:33:58.473261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d7bc2-3dce-4157-b03a-8c10c1e51737 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:58.491195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d7bc2-3dce-4157-b03a-8c10c1e51737 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:58.503726 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95931059-0638-4853-aa98-1d6ee453c7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:58.504839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95931059-0638-4853-aa98-1d6ee453c7d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:58.505149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95931059-0638-4853-aa98-1d6ee453c7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:58.505663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95931059-0638-4853-aa98-1d6ee453c7d7 admin admin] PATH_INFO: `/v3/projects/e518b4f271f647409fb12d891c4579a7/users/d0772f709b1940f9a5991b7cc41547df/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:58.523827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95931059-0638-4853-aa98-1d6ee453c7d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e518b4f271f647409fb12d891c4579a7, user_id=d0772f709b1940f9a5991b7cc41547df, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:58.527050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95931059-0638-4853-aa98-1d6ee453c7d7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:58.549303 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1229/2452] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:33:58 2022] PUT /identity/v3/projects/e518b4f271f647409fb12d891c4579a7/users/d0772f709b1940f9a5991b7cc41547df/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:58.553555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f873d37-8d48-4332-90f3-30b8e2bf11ce admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:58.568325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f873d37-8d48-4332-90f3-30b8e2bf11ce admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:58.707663 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22e79b-bcf7-455a-804c-bb812d80c086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:58.708610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22e79b-bcf7-455a-804c-bb812d80c086 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:58.709007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22e79b-bcf7-455a-804c-bb812d80c086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:58.709313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22e79b-bcf7-455a-804c-bb812d80c086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:58.710341 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22e79b-bcf7-455a-804c-bb812d80c086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:58.712470 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22e79b-bcf7-455a-804c-bb812d80c086 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:58.728257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1224/2453] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:33:58.733537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95931059-0638-4853-aa98-1d6ee453c7d7 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:58.764294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95931059-0638-4853-aa98-1d6ee453c7d7 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:58.773556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759d0c-5620-42a9-9365-6be3792f25a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:58.774997 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759d0c-5620-42a9-9365-6be3792f25a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:58.775325 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759d0c-5620-42a9-9365-6be3792f25a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:58.775804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759d0c-5620-42a9-9365-6be3792f25a8 admin admin] PATH_INFO: `/v3/projects/e518b4f271f647409fb12d891c4579a7/users/d0772f709b1940f9a5991b7cc41547df/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:58.778440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759d0c-5620-42a9-9365-6be3792f25a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e518b4f271f647409fb12d891c4579a7, user_id=d0772f709b1940f9a5991b7cc41547df, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:58.782680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759d0c-5620-42a9-9365-6be3792f25a8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:58.798781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1230/2454] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:33:58 2022] PUT /identity/v3/projects/e518b4f271f647409fb12d891c4579a7/users/d0772f709b1940f9a5991b7cc41547df/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:33:58.809582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc835091-e4c3-47a4-9070-40ddbfc8492f None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:58.810750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc835091-e4c3-47a4-9070-40ddbfc8492f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:58.811935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc835091-e4c3-47a4-9070-40ddbfc8492f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:58.864049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-dc835091-e4c3-47a4-9070-40ddbfc8492f None None] MFA Rules not processed for user `d0772f709b1940f9a5991b7cc41547df`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:33:58.865780 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-dc835091-e4c3-47a4-9070-40ddbfc8492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:59.021165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1225/2455] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:33:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:33:59.036086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759d0c-5620-42a9-9365-6be3792f25a8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:59.038560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f7759d0c-5620-42a9-9365-6be3792f25a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:59.120276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:59.136681 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759d0c-5620-42a9-9365-6be3792f25a8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:59.141199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e053df1-56a5-4c0f-a590-a574adac5734 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:59.141973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e053df1-56a5-4c0f-a590-a574adac5734 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:59.142724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e053df1-56a5-4c0f-a590-a574adac5734 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:59.142724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e053df1-56a5-4c0f-a590-a574adac5734 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:59.153786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e053df1-56a5-4c0f-a590-a574adac5734 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:59.156386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e053df1-56a5-4c0f-a590-a574adac5734 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:59.181983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1231/2456] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:33:59.290243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc835091-e4c3-47a4-9070-40ddbfc8492f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:33:59.292689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-dc835091-e4c3-47a4-9070-40ddbfc8492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:33:59.320077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc835091-e4c3-47a4-9070-40ddbfc8492f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:33:59.332009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348ef7-3692-4570-9c84-6389f6e1f956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:33:59.335294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ef7-3692-4570-9c84-6389f6e1f956 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:33:59.335645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ef7-3692-4570-9c84-6389f6e1f956 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:33:59.335860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348ef7-3692-4570-9c84-6389f6e1f956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:33:59.485761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:33:59.492428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3348ef7-3692-4570-9c84-6389f6e1f956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:33:59.496071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3348ef7-3692-4570-9c84-6389f6e1f956 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:33:59.537870 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1226/2457] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:33:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:00.484593 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e053df1-56a5-4c0f-a590-a574adac5734 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:00.491874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2e053df1-56a5-4c0f-a590-a574adac5734 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:00.584329 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e053df1-56a5-4c0f-a590-a574adac5734 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:00.589064 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:00.589931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:00.590206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:00.590475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] PATH_INFO: `/v3/users/8632496bdc1b4d06a74caf483c8b0a42` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:00.591713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8632496bdc1b4d06a74caf483c8b0a42)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:00.593646 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:00.703363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8632496bdc1b4d06a74caf483c8b0a42'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:00.725011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8632496bdc1b4d06a74caf483c8b0a42'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:00.725783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8632496bdc1b4d06a74caf483c8b0a42'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:00.740318 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8632496bdc1b4d06a74caf483c8b0a42'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:00.741258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8632496bdc1b4d06a74caf483c8b0a42'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:00.773707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1232/2458] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:00 2022] DELETE /identity/v3/users/8632496bdc1b4d06a74caf483c8b0a42 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:01.058024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348ef7-3692-4570-9c84-6389f6e1f956 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:01.060692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c3348ef7-3692-4570-9c84-6389f6e1f956 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:01.187476 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:01.188924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:01.272773 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348ef7-3692-4570-9c84-6389f6e1f956 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:01.332495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc46d15-cbb7-495d-8bf4-4f4627f6412b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:01.338594 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea500386-c679-45da-b567-31d8db51e5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:01.340349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500386-c679-45da-b567-31d8db51e5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:01.340860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500386-c679-45da-b567-31d8db51e5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:01.341845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea500386-c679-45da-b567-31d8db51e5d5 admin admin] PATH_INFO: `/v3/users/781d4a4643374bc6a78c3553d62f7672` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:01.343157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea500386-c679-45da-b567-31d8db51e5d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=781d4a4643374bc6a78c3553d62f7672)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:01.345774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea500386-c679-45da-b567-31d8db51e5d5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:01.355138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:01.387695 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d3e37-85b8-4aeb-a962-fb298333ee16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:01.389371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d3e37-85b8-4aeb-a962-fb298333ee16 service placement] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:01.389809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d3e37-85b8-4aeb-a962-fb298333ee16 service placement] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:01.390274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d3e37-85b8-4aeb-a962-fb298333ee16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:01.393250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e36d3e37-85b8-4aeb-a962-fb298333ee16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:01.408631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d3e37-85b8-4aeb-a962-fb298333ee16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:01.410449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d3e37-85b8-4aeb-a962-fb298333ee16 service placement] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:01.424610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1227/2459] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:01.425889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d3e37-85b8-4aeb-a962-fb298333ee16 service placement] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:01.427611 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e36d3e37-85b8-4aeb-a962-fb298333ee16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:01.428885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ea500386-c679-45da-b567-31d8db51e5d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '781d4a4643374bc6a78c3553d62f7672'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.446449 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d3e37-85b8-4aeb-a962-fb298333ee16 service placement] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:01.458227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:01.459419 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:01.459541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ea500386-c679-45da-b567-31d8db51e5d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '781d4a4643374bc6a78c3553d62f7672'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.459835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:01.460157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ea500386-c679-45da-b567-31d8db51e5d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '781d4a4643374bc6a78c3553d62f7672'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.460269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] PATH_INFO: `/v3/projects/16edd3f309d54728882acc6d23f8861e` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:01.461844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16edd3f309d54728882acc6d23f8861e)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:01.465486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:01.474892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ea500386-c679-45da-b567-31d8db51e5d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '781d4a4643374bc6a78c3553d62f7672'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.475676 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ea500386-c679-45da-b567-31d8db51e5d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '781d4a4643374bc6a78c3553d62f7672'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.494918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1233/2460] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:01 2022] DELETE /identity/v3/users/781d4a4643374bc6a78c3553d62f7672 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:01.556271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16edd3f309d54728882acc6d23f8861e', 'request_id': 'req-33abe5a6-94d2-4739-b497-6e578d379c79'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.570861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16edd3f309d54728882acc6d23f8861e', 'request_id': 'req-33abe5a6-94d2-4739-b497-6e578d379c79'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.580153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16edd3f309d54728882acc6d23f8861e', 'request_id': 'req-33abe5a6-94d2-4739-b497-6e578d379c79'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.592595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] The token cache is being invalidate because project 16edd3f309d54728882acc6d23f8861e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:01.592891 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.593566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-33abe5a6-94d2-4739-b497-6e578d379c79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.598453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1228/2461] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:01 2022] DELETE /identity/v3/projects/16edd3f309d54728882acc6d23f8861e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:01.601273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7204fb7-676a-4b66-b67b-41a3c57e60ec None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:01.601567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7204fb7-676a-4b66-b67b-41a3c57e60ec None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:01.602579 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7204fb7-676a-4b66-b67b-41a3c57e60ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:01.622183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c7204fb7-676a-4b66-b67b-41a3c57e60ec None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:01.624027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c7204fb7-676a-4b66-b67b-41a3c57e60ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:01.693917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1234/2462] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:01.702864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039d9a9-de91-4d8d-9750-60c72929880d None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:01.705746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039d9a9-de91-4d8d-9750-60c72929880d None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:01.706044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039d9a9-de91-4d8d-9750-60c72929880d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:01.707528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7204fb7-676a-4b66-b67b-41a3c57e60ec None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:01.708989 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c7204fb7-676a-4b66-b67b-41a3c57e60ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:01.721145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7204fb7-676a-4b66-b67b-41a3c57e60ec None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:01.726098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:01.727004 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:01.727292 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:01.727572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] PATH_INFO: `/v3/projects/98f97ac03be44e0bb8ab21d985d1dd27` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:01.728798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f97ac03be44e0bb8ab21d985d1dd27)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:01.730377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-6039d9a9-de91-4d8d-9750-60c72929880d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:01.731745 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6039d9a9-de91-4d8d-9750-60c72929880d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:01.732092 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:01.752034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1229/2463] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:01.756838 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039d9a9-de91-4d8d-9750-60c72929880d None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:01.764061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:01.768190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039d9a9-de91-4d8d-9750-60c72929880d None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:01.774933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038afe51-878d-4425-a300-2300c0d4f9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:01.775687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038afe51-878d-4425-a300-2300c0d4f9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:01.775992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038afe51-878d-4425-a300-2300c0d4f9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:01.776985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038afe51-878d-4425-a300-2300c0d4f9f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:01.778608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038afe51-878d-4425-a300-2300c0d4f9f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:01.779729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038afe51-878d-4425-a300-2300c0d4f9f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:01.781418 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038afe51-878d-4425-a300-2300c0d4f9f7 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:01.790914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f97ac03be44e0bb8ab21d985d1dd27', 'request_id': 'req-e0eceb61-1804-4533-818c-18ca1e516909'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.794234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1230/2464] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:34:01.797356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f97ac03be44e0bb8ab21d985d1dd27', 'request_id': 'req-e0eceb61-1804-4533-818c-18ca1e516909'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.799588 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038afe51-878d-4425-a300-2300c0d4f9f7 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:01.807861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f97ac03be44e0bb8ab21d985d1dd27', 'request_id': 'req-e0eceb61-1804-4533-818c-18ca1e516909'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.811321 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038afe51-878d-4425-a300-2300c0d4f9f7 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:01.824112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] The token cache is being invalidate because project 98f97ac03be44e0bb8ab21d985d1dd27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:01.824706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.825459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:01.827617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1235/2465] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:01 2022] DELETE /identity/v3/projects/98f97ac03be44e0bb8ab21d985d1dd27 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:01.831846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:01.833222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:01.844898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:01.884467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4016df-0459-4209-9a40-c6bb2d08703c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:01.885968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4016df-0459-4209-9a40-c6bb2d08703c admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:01.886263 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4016df-0459-4209-9a40-c6bb2d08703c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:01.886575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4016df-0459-4209-9a40-c6bb2d08703c admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:01.887291 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4016df-0459-4209-9a40-c6bb2d08703c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:01.890880 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4016df-0459-4209-9a40-c6bb2d08703c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:01.914320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eceb61-1804-4533-818c-18ca1e516909 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:01.921074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:01.922255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:01.922635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:01.922914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] PATH_INFO: `/v3/users/e2c24c560e4045a38156ad50757ad7dc` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:01.924185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2c24c560e4045a38156ad50757ad7dc)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:01.926443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:01.934549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1231/2466] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:01 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:01.939328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4016df-0459-4209-9a40-c6bb2d08703c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:01.941430 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7c4016df-0459-4209-9a40-c6bb2d08703c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:02.004366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2c24c560e4045a38156ad50757ad7dc'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:02.023737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2c24c560e4045a38156ad50757ad7dc'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:02.024798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2c24c560e4045a38156ad50757ad7dc'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:02.032902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2c24c560e4045a38156ad50757ad7dc'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:02.034202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2c24c560e4045a38156ad50757ad7dc'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:02.041596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4016df-0459-4209-9a40-c6bb2d08703c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.049024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817e99f-593d-438b-bed9-eeaad5301b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.049904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817e99f-593d-438b-bed9-eeaad5301b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.050340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817e99f-593d-438b-bed9-eeaad5301b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.050822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817e99f-593d-438b-bed9-eeaad5301b51 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.051524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1236/2467] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:01 2022] DELETE /identity/v3/users/e2c24c560e4045a38156ad50757ad7dc => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:02.051888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817e99f-593d-438b-bed9-eeaad5301b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.054261 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817e99f-593d-438b-bed9-eeaad5301b51 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.073823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:02.076190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:02.081623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1232/2468] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:01 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:02.082684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817e99f-593d-438b-bed9-eeaad5301b51 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:02.085000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9817e99f-593d-438b-bed9-eeaad5301b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:02.102427 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817e99f-593d-438b-bed9-eeaad5301b51 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.144694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2eba5d-7b68-4dcb-979f-bf96ecca33a8 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.150108 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74a797-e3d6-4e42-835a-95e6d1f27cbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.151123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74a797-e3d6-4e42-835a-95e6d1f27cbd service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.151608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74a797-e3d6-4e42-835a-95e6d1f27cbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.151887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74a797-e3d6-4e42-835a-95e6d1f27cbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.155434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5e74a797-e3d6-4e42-835a-95e6d1f27cbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:02.164926 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688880a-7ca6-411d-90af-30c822521d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.165952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688880a-7ca6-411d-90af-30c822521d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.166652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688880a-7ca6-411d-90af-30c822521d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.166652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688880a-7ca6-411d-90af-30c822521d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.167439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688880a-7ca6-411d-90af-30c822521d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.169798 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688880a-7ca6-411d-90af-30c822521d6b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.182018 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1233/2469] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:02.186433 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688880a-7ca6-411d-90af-30c822521d6b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:02.193143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:02.199230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688880a-7ca6-411d-90af-30c822521d6b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.205466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff769e-ace1-400b-bef1-98ea18464cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.206146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff769e-ace1-400b-bef1-98ea18464cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.206425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff769e-ace1-400b-bef1-98ea18464cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.206728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff769e-ace1-400b-bef1-98ea18464cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.207456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff769e-ace1-400b-bef1-98ea18464cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.209093 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff769e-ace1-400b-bef1-98ea18464cfe admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.210288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:02.228840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1234/2470] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:02.229889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff769e-ace1-400b-bef1-98ea18464cfe admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:02.242205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff769e-ace1-400b-bef1-98ea18464cfe admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.248003 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fc246-6591-445b-9beb-74b0161f08ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.249099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fc246-6591-445b-9beb-74b0161f08ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.249389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fc246-6591-445b-9beb-74b0161f08ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.249665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fc246-6591-445b-9beb-74b0161f08ff admin admin] PATH_INFO: `/v3/projects/8a0935e619f143e58d185db5a8a581f1/users/e3552aa56aa6467190ac54fb39cb7a82/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.251220 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74a797-e3d6-4e42-835a-95e6d1f27cbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.252580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74a797-e3d6-4e42-835a-95e6d1f27cbd service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.272227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fc246-6591-445b-9beb-74b0161f08ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a0935e619f143e58d185db5a8a581f1, user_id=e3552aa56aa6467190ac54fb39cb7a82, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.283573 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1237/2471] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:34:02.283856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74a797-e3d6-4e42-835a-95e6d1f27cbd service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:02.284790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fc246-6591-445b-9beb-74b0161f08ff admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.287476 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5e74a797-e3d6-4e42-835a-95e6d1f27cbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:02.303896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74a797-e3d6-4e42-835a-95e6d1f27cbd service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.314625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1235/2472] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:02 2022] PUT /identity/v3/projects/8a0935e619f143e58d185db5a8a581f1/users/e3552aa56aa6467190ac54fb39cb7a82/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:02.320141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce1d0e-6594-4db6-8fd6-8c318eaeda9b None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.320330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce1d0e-6594-4db6-8fd6-8c318eaeda9b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.320816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce1d0e-6594-4db6-8fd6-8c318eaeda9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.354770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-06ce1d0e-6594-4db6-8fd6-8c318eaeda9b None None] MFA Rules not processed for user `e3552aa56aa6467190ac54fb39cb7a82`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:02.356072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-06ce1d0e-6594-4db6-8fd6-8c318eaeda9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:02.376510 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.377773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.377773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.379845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] PATH_INFO: `/v3/projects/a122683898334dc5b09dc4f24b9370b1` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.379845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a122683898334dc5b09dc4f24b9370b1)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.381948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.435478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1236/2473] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:02.440109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce1d0e-6594-4db6-8fd6-8c318eaeda9b None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:02.443536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a122683898334dc5b09dc4f24b9370b1', 'request_id': 'req-a97b5ffc-d59e-4106-a687-ce398b301200'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:02.450047 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a122683898334dc5b09dc4f24b9370b1', 'request_id': 'req-a97b5ffc-d59e-4106-a687-ce398b301200'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:02.451398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce1d0e-6594-4db6-8fd6-8c318eaeda9b None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.459370 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a122683898334dc5b09dc4f24b9370b1', 'request_id': 'req-a97b5ffc-d59e-4106-a687-ce398b301200'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:02.474850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] The token cache is being invalidate because project a122683898334dc5b09dc4f24b9370b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:02.475539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:02.476242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:02.478610 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1238/2474] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:02 2022] DELETE /identity/v3/projects/a122683898334dc5b09dc4f24b9370b1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:02.523674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4ef54-c932-4b5f-aa78-7a233a8927a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.523907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4ef54-c932-4b5f-aa78-7a233a8927a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.524172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4ef54-c932-4b5f-aa78-7a233a8927a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.524365 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf4ef54-c932-4b5f-aa78-7a233a8927a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.525089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf4ef54-c932-4b5f-aa78-7a233a8927a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.527613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf4ef54-c932-4b5f-aa78-7a233a8927a8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.570040 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1237/2475] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:02 2022] POST /identity/v3/projects => generated 369 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:02.572231 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:02.575784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:02.599378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b5ffc-d59e-4106-a687-ce398b301200 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.711868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab9570-165e-4ddb-b060-653803dbc8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.712725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab9570-165e-4ddb-b060-653803dbc8fa admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.713092 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab9570-165e-4ddb-b060-653803dbc8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.713447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab9570-165e-4ddb-b060-653803dbc8fa admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.714420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab9570-165e-4ddb-b060-653803dbc8fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.718218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab9570-165e-4ddb-b060-653803dbc8fa admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.738931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:02.751087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1239/2476] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:02 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:02.755410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4ef54-c932-4b5f-aa78-7a233a8927a8 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:02.767752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf4ef54-c932-4b5f-aa78-7a233a8927a8 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.772429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167b9d3-2f3b-4c44-9c3b-18be8a5bcde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.773104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167b9d3-2f3b-4c44-9c3b-18be8a5bcde1 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.773401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167b9d3-2f3b-4c44-9c3b-18be8a5bcde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.773661 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167b9d3-2f3b-4c44-9c3b-18be8a5bcde1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.774451 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167b9d3-2f3b-4c44-9c3b-18be8a5bcde1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.776072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1167b9d3-2f3b-4c44-9c3b-18be8a5bcde1 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.790055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1238/2477] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:02.794965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab9570-165e-4ddb-b060-653803dbc8fa admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:02.805479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab9570-165e-4ddb-b060-653803dbc8fa admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.810718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a547f9e-fadf-42a7-a7fb-b49565c24042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.811745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a547f9e-fadf-42a7-a7fb-b49565c24042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.812106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a547f9e-fadf-42a7-a7fb-b49565c24042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.812472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a547f9e-fadf-42a7-a7fb-b49565c24042 admin admin] PATH_INFO: `/v3/projects/265dcad1c93b40d58c4e0128efc523f8/users/1b9aa67988ca414993a9dcc485afeab6/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.830483 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a547f9e-fadf-42a7-a7fb-b49565c24042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=265dcad1c93b40d58c4e0128efc523f8, user_id=1b9aa67988ca414993a9dcc485afeab6, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.833248 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a547f9e-fadf-42a7-a7fb-b49565c24042 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.850568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1240/2478] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:02 2022] PUT /identity/v3/projects/265dcad1c93b40d58c4e0128efc523f8/users/1b9aa67988ca414993a9dcc485afeab6/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:02.852304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167b9d3-2f3b-4c44-9c3b-18be8a5bcde1 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:02.862854 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167b9d3-2f3b-4c44-9c3b-18be8a5bcde1 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.896154 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:02.930732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012f7aa2-1fde-4bb4-b0a1-8a802b5315fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.931412 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f7aa2-1fde-4bb4-b0a1-8a802b5315fa admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.931709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f7aa2-1fde-4bb4-b0a1-8a802b5315fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.931987 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012f7aa2-1fde-4bb4-b0a1-8a802b5315fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.932730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012f7aa2-1fde-4bb4-b0a1-8a802b5315fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.934334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012f7aa2-1fde-4bb4-b0a1-8a802b5315fa admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.946583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1239/2479] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:02.950071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a547f9e-fadf-42a7-a7fb-b49565c24042 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:02.961144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a547f9e-fadf-42a7-a7fb-b49565c24042 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:02.966680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f769d-bd10-41e4-ac81-ffa10352ae22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:02.967609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f769d-bd10-41e4-ac81-ffa10352ae22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:02.967897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f769d-bd10-41e4-ac81-ffa10352ae22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:02.968178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f769d-bd10-41e4-ac81-ffa10352ae22 admin admin] PATH_INFO: `/v3/projects/265dcad1c93b40d58c4e0128efc523f8/users/1b9aa67988ca414993a9dcc485afeab6/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:02.970349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f769d-bd10-41e4-ac81-ffa10352ae22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=265dcad1c93b40d58c4e0128efc523f8, user_id=1b9aa67988ca414993a9dcc485afeab6, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:02.973139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099f769d-bd10-41e4-ac81-ffa10352ae22 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:02.984333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1241/2480] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:02 2022] PUT /identity/v3/projects/265dcad1c93b40d58c4e0128efc523f8/users/1b9aa67988ca414993a9dcc485afeab6/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:02.999597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435071-6e1e-4d0e-ab12-34b1973f8365 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:03.000070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435071-6e1e-4d0e-ab12-34b1973f8365 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:03.000263 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f435071-6e1e-4d0e-ab12-34b1973f8365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:03.031058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-9f435071-6e1e-4d0e-ab12-34b1973f8365 None None] MFA Rules not processed for user `1b9aa67988ca414993a9dcc485afeab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:03.032806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9f435071-6e1e-4d0e-ab12-34b1973f8365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:03.105064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1240/2481] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:02 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:03.116727 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f769d-bd10-41e4-ac81-ffa10352ae22 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:03.118408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-099f769d-bd10-41e4-ac81-ffa10352ae22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:03.180718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099f769d-bd10-41e4-ac81-ffa10352ae22 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:03.184401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5873e-a499-4dbf-abd2-7c302586ff9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:03.185683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5873e-a499-4dbf-abd2-7c302586ff9c service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:03.186106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5873e-a499-4dbf-abd2-7c302586ff9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:03.186395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5873e-a499-4dbf-abd2-7c302586ff9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:03.195697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5873e-a499-4dbf-abd2-7c302586ff9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:03.197283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5873e-a499-4dbf-abd2-7c302586ff9c service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:03.210669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1242/2482] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:03 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:03.283253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f435071-6e1e-4d0e-ab12-34b1973f8365 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:03.293673 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f435071-6e1e-4d0e-ab12-34b1973f8365 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:03.307599 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacf897-95fb-4690-aa4d-a5ba59998c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:03.308311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacf897-95fb-4690-aa4d-a5ba59998c48 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:03.308597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacf897-95fb-4690-aa4d-a5ba59998c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:03.308877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacf897-95fb-4690-aa4d-a5ba59998c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:03.312164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6dacf897-95fb-4690-aa4d-a5ba59998c48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:03.394458 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacf897-95fb-4690-aa4d-a5ba59998c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:03.396028 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacf897-95fb-4690-aa4d-a5ba59998c48 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:03.407964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1241/2483] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:05.708715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5873e-a499-4dbf-abd2-7c302586ff9c service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:05.710162 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3cc5873e-a499-4dbf-abd2-7c302586ff9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:05.779986 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5873e-a499-4dbf-abd2-7c302586ff9c service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:05.784596 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8072db-b8d9-404f-8ed4-232123e0febc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:05.785461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8072db-b8d9-404f-8ed4-232123e0febc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:05.785741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8072db-b8d9-404f-8ed4-232123e0febc admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:05.786015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8072db-b8d9-404f-8ed4-232123e0febc admin admin] PATH_INFO: `/v3/users/47d7587d10b3487688ed251a85b83c97` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:05.787215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8072db-b8d9-404f-8ed4-232123e0febc admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d7587d10b3487688ed251a85b83c97)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:05.789288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8072db-b8d9-404f-8ed4-232123e0febc admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:05.836993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-de8072db-b8d9-404f-8ed4-232123e0febc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d7587d10b3487688ed251a85b83c97'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:05.850055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-de8072db-b8d9-404f-8ed4-232123e0febc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d7587d10b3487688ed251a85b83c97'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:05.851196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-de8072db-b8d9-404f-8ed4-232123e0febc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d7587d10b3487688ed251a85b83c97'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:05.852206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:05.861005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-de8072db-b8d9-404f-8ed4-232123e0febc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d7587d10b3487688ed251a85b83c97'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:05.861601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-de8072db-b8d9-404f-8ed4-232123e0febc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d7587d10b3487688ed251a85b83c97'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:05.879042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1243/2484] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:05 2022] DELETE /identity/v3/users/47d7587d10b3487688ed251a85b83c97 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:06.073061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacf897-95fb-4690-aa4d-a5ba59998c48 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:06.085809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacf897-95fb-4690-aa4d-a5ba59998c48 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:06.150151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:06.165150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:06.165902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:06.166195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:06.166471 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] PATH_INFO: `/v3/projects/1b930cadb2dc4e49947db0514528617d` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:06.167686 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b930cadb2dc4e49947db0514528617d)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:06.170616 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:06.217901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b930cadb2dc4e49947db0514528617d', 'request_id': 'req-276e3fe9-e827-4360-bac6-8ed948cf400d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:06.223304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b930cadb2dc4e49947db0514528617d', 'request_id': 'req-276e3fe9-e827-4360-bac6-8ed948cf400d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:06.230760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b930cadb2dc4e49947db0514528617d', 'request_id': 'req-276e3fe9-e827-4360-bac6-8ed948cf400d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:06.241583 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] The token cache is being invalidate because project 1b930cadb2dc4e49947db0514528617d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:06.241869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:06.242496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-276e3fe9-e827-4360-bac6-8ed948cf400d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:06.244072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1242/2485] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:06 2022] DELETE /identity/v3/projects/1b930cadb2dc4e49947db0514528617d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:06.249430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63534f-53c1-44c3-a9c9-7705888f79fd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:06.249723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63534f-53c1-44c3-a9c9-7705888f79fd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:06.250000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63534f-53c1-44c3-a9c9-7705888f79fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:06.269334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7a63534f-53c1-44c3-a9c9-7705888f79fd None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:06.270567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7a63534f-53c1-44c3-a9c9-7705888f79fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:06.329882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1244/2486] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:06.337656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0208e9-c8d9-4682-836f-2101e9094ab5 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:06.337948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0208e9-c8d9-4682-836f-2101e9094ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:06.338232 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0208e9-c8d9-4682-836f-2101e9094ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:06.360988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-ae0208e9-c8d9-4682-836f-2101e9094ab5 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:06.363149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ae0208e9-c8d9-4682-836f-2101e9094ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:06.374639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1243/2487] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:06.378966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63534f-53c1-44c3-a9c9-7705888f79fd None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:06.390197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63534f-53c1-44c3-a9c9-7705888f79fd None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:06.394799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5bbf3-c296-4caa-8091-0f00a2fb577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:06.395463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5bbf3-c296-4caa-8091-0f00a2fb577a admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:06.395747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5bbf3-c296-4caa-8091-0f00a2fb577a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:06.396023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5bbf3-c296-4caa-8091-0f00a2fb577a admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:06.396744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5bbf3-c296-4caa-8091-0f00a2fb577a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:06.397049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5bbf3-c296-4caa-8091-0f00a2fb577a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:06.398699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5bbf3-c296-4caa-8091-0f00a2fb577a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:06.410328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1245/2488] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:34:06.414864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0208e9-c8d9-4682-836f-2101e9094ab5 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:06.428000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0208e9-c8d9-4682-836f-2101e9094ab5 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:06.436356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf930c-cd85-4f62-9819-44971d327660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:06.437007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf930c-cd85-4f62-9819-44971d327660 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:06.437295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf930c-cd85-4f62-9819-44971d327660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:06.437574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf930c-cd85-4f62-9819-44971d327660 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:06.438276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf930c-cd85-4f62-9819-44971d327660 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:06.440237 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdf930c-cd85-4f62-9819-44971d327660 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:06.465450 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1244/2489] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:06 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:06.469882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5bbf3-c296-4caa-8091-0f00a2fb577a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:06.479785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5bbf3-c296-4caa-8091-0f00a2fb577a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:06.537718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7a32f-fcae-4897-8c1c-1cfc03ca11de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:06.538392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7a32f-fcae-4897-8c1c-1cfc03ca11de admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:06.538798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7a32f-fcae-4897-8c1c-1cfc03ca11de admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:06.539162 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7a32f-fcae-4897-8c1c-1cfc03ca11de admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:06.539853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7a32f-fcae-4897-8c1c-1cfc03ca11de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:06.541792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7a32f-fcae-4897-8c1c-1cfc03ca11de admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:06.566753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1246/2490] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:06 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:06.568331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf930c-cd85-4f62-9819-44971d327660 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:06.579126 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf930c-cd85-4f62-9819-44971d327660 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:06.584049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc46a4-8b50-4652-a0fe-887a183aa2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:06.584833 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc46a4-8b50-4652-a0fe-887a183aa2dc admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:06.585198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc46a4-8b50-4652-a0fe-887a183aa2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:06.585538 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc46a4-8b50-4652-a0fe-887a183aa2dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:06.587065 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc46a4-8b50-4652-a0fe-887a183aa2dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:06.589499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbc46a4-8b50-4652-a0fe-887a183aa2dc admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:06.600538 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1245/2491] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:06.604663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7a32f-fcae-4897-8c1c-1cfc03ca11de admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:06.615570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7a32f-fcae-4897-8c1c-1cfc03ca11de admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:06.620207 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ffe11e-40c2-47bf-8588-62be409f4551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:06.620864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe11e-40c2-47bf-8588-62be409f4551 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:06.621144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe11e-40c2-47bf-8588-62be409f4551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:06.621417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ffe11e-40c2-47bf-8588-62be409f4551 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:06.622759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffe11e-40c2-47bf-8588-62be409f4551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:06.624379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ffe11e-40c2-47bf-8588-62be409f4551 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:06.634494 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1247/2492] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:06.638981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc46a4-8b50-4652-a0fe-887a183aa2dc admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:06.649078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc46a4-8b50-4652-a0fe-887a183aa2dc admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:06.653837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2c0ff-2929-438c-a660-6e840280db5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:06.654750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2c0ff-2929-438c-a660-6e840280db5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:06.655029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2c0ff-2929-438c-a660-6e840280db5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:06.655329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2c0ff-2929-438c-a660-6e840280db5e admin admin] PATH_INFO: `/v3/projects/2428a60e3a0243c1a250ecb046138e36/users/325db81829e946ef8142c21466992d93/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:06.672795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2c0ff-2929-438c-a660-6e840280db5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2428a60e3a0243c1a250ecb046138e36, user_id=325db81829e946ef8142c21466992d93, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:06.675761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2c0ff-2929-438c-a660-6e840280db5e admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:06.691703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1246/2493] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:06 2022] PUT /identity/v3/projects/2428a60e3a0243c1a250ecb046138e36/users/325db81829e946ef8142c21466992d93/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:06.697246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca5b20-63e1-4285-854a-0cbc43a4ad6a None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:06.697671 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca5b20-63e1-4285-854a-0cbc43a4ad6a None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:06.697949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca5b20-63e1-4285-854a-0cbc43a4ad6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:06.718404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-09ca5b20-63e1-4285-854a-0cbc43a4ad6a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:06.719641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-09ca5b20-63e1-4285-854a-0cbc43a4ad6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:06.769445 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:06.788770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1248/2494] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:06.801371 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2c0ff-2929-438c-a660-6e840280db5e admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:06.803253 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6ed2c0ff-2929-438c-a660-6e840280db5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:06.864681 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2c0ff-2929-438c-a660-6e840280db5e admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:06.868632 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0d59e-1199-4545-8ed0-e9e5dee37aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:06.869279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0d59e-1199-4545-8ed0-e9e5dee37aac service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:06.869564 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0d59e-1199-4545-8ed0-e9e5dee37aac service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:06.869843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0d59e-1199-4545-8ed0-e9e5dee37aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:06.879128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0d59e-1199-4545-8ed0-e9e5dee37aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:06.880589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0d59e-1199-4545-8ed0-e9e5dee37aac service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:06.890368 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1247/2495] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:07.585390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c0df8-8c3b-489a-898d-2fe29cb4c4d0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:07.585687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c0df8-8c3b-489a-898d-2fe29cb4c4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:07.585956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c0df8-8c3b-489a-898d-2fe29cb4c4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:07.635427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c35c0df8-8c3b-489a-898d-2fe29cb4c4d0 None None] MFA Rules not processed for user `325db81829e946ef8142c21466992d93`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:07.637151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c35c0df8-8c3b-489a-898d-2fe29cb4c4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:07.762614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1249/2496] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:07.767478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0d59e-1199-4545-8ed0-e9e5dee37aac service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:07.777483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0d59e-1199-4545-8ed0-e9e5dee37aac service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:07.783606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d21805-fdfc-4495-8387-c1666bfa9a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:07.784543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d21805-fdfc-4495-8387-c1666bfa9a3a admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:07.785031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d21805-fdfc-4495-8387-c1666bfa9a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:07.785511 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d21805-fdfc-4495-8387-c1666bfa9a3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:07.786418 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d21805-fdfc-4495-8387-c1666bfa9a3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:07.788902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d21805-fdfc-4495-8387-c1666bfa9a3a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:07.811173 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:07.822792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1248/2497] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:07 2022] POST /identity/v3/projects => generated 405 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:07.827070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c0df8-8c3b-489a-898d-2fe29cb4c4d0 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:07.840835 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c0df8-8c3b-489a-898d-2fe29cb4c4d0 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:07.903152 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7bead-91b2-459c-a630-e58744e78260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:07.903888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7bead-91b2-459c-a630-e58744e78260 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:07.904171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7bead-91b2-459c-a630-e58744e78260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:07.904587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7bead-91b2-459c-a630-e58744e78260 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:07.905260 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7bead-91b2-459c-a630-e58744e78260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:07.907246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7bead-91b2-459c-a630-e58744e78260 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:07.931507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1250/2498] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:07 2022] POST /identity/v3/users => generated 309 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:07.933273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d21805-fdfc-4495-8387-c1666bfa9a3a admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:07.945759 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d21805-fdfc-4495-8387-c1666bfa9a3a admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:07.955445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5a9d-dc8b-4123-a8c7-46a54938932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:07.955445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5a9d-dc8b-4123-a8c7-46a54938932b admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:07.955445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5a9d-dc8b-4123-a8c7-46a54938932b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:07.955445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef5a9d-dc8b-4123-a8c7-46a54938932b admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:07.955952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef5a9d-dc8b-4123-a8c7-46a54938932b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:07.955952 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ef5a9d-dc8b-4123-a8c7-46a54938932b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:07.967105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1249/2499] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:07.971736 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7bead-91b2-459c-a630-e58744e78260 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:07.984107 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7bead-91b2-459c-a630-e58744e78260 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:07.989239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2aca4-4baf-4a26-bb68-3b580722c99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:07.990184 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2aca4-4baf-4a26-bb68-3b580722c99f admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:07.990465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2aca4-4baf-4a26-bb68-3b580722c99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:07.990780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2aca4-4baf-4a26-bb68-3b580722c99f admin admin] PATH_INFO: `/v3/projects/80636d08a25f49ce85206d054da815b5/users/77256c08ae7c42c3acc754f691d7ca91/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:08.006641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2aca4-4baf-4a26-bb68-3b580722c99f admin admin] RBAC: Authorizing `identity:create_grant(project_id=80636d08a25f49ce85206d054da815b5, user_id=77256c08ae7c42c3acc754f691d7ca91, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:08.009427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2aca4-4baf-4a26-bb68-3b580722c99f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:08.027144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1251/2500] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:07 2022] PUT /identity/v3/projects/80636d08a25f49ce85206d054da815b5/users/77256c08ae7c42c3acc754f691d7ca91/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:08.031443 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5a9d-dc8b-4123-a8c7-46a54938932b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:08.042464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef5a9d-dc8b-4123-a8c7-46a54938932b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:08.100898 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaebbe-5ae3-4740-949f-b9ddb67d7af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:08.101706 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaebbe-5ae3-4740-949f-b9ddb67d7af0 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:08.101989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaebbe-5ae3-4740-949f-b9ddb67d7af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:08.102264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaebbe-5ae3-4740-949f-b9ddb67d7af0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:08.103033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aaebbe-5ae3-4740-949f-b9ddb67d7af0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:08.104634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aaebbe-5ae3-4740-949f-b9ddb67d7af0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:08.114991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1250/2501] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:08.119190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2aca4-4baf-4a26-bb68-3b580722c99f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:08.130206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2aca4-4baf-4a26-bb68-3b580722c99f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:08.135281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c04a45-d893-41d9-ad09-4736d1dc9278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:08.136159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c04a45-d893-41d9-ad09-4736d1dc9278 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:08.136439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c04a45-d893-41d9-ad09-4736d1dc9278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:08.136714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c04a45-d893-41d9-ad09-4736d1dc9278 admin admin] PATH_INFO: `/v3/projects/80636d08a25f49ce85206d054da815b5/users/77256c08ae7c42c3acc754f691d7ca91/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:08.138732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c04a45-d893-41d9-ad09-4736d1dc9278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80636d08a25f49ce85206d054da815b5, user_id=77256c08ae7c42c3acc754f691d7ca91, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:08.141399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c04a45-d893-41d9-ad09-4736d1dc9278 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:08.151875 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1252/2502] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:08 2022] PUT /identity/v3/projects/80636d08a25f49ce85206d054da815b5/users/77256c08ae7c42c3acc754f691d7ca91/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:08.824478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5d90e-e7ed-4520-9be9-22b12c84b6c7 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:08.826037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5d90e-e7ed-4520-9be9-22b12c84b6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:08.826344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5d90e-e7ed-4520-9be9-22b12c84b6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:08.858498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-9aa5d90e-e7ed-4520-9be9-22b12c84b6c7 None None] MFA Rules not processed for user `77256c08ae7c42c3acc754f691d7ca91`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:08.859742 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-9aa5d90e-e7ed-4520-9be9-22b12c84b6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:08.933027 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1251/2503] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:08 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:10.037940 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c04a45-d893-41d9-ad09-4736d1dc9278 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:10.074829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:10.107161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c04a45-d893-41d9-ad09-4736d1dc9278 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:10.112471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:10.113401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:10.113731 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:10.114054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] PATH_INFO: `/v3/users/325db81829e946ef8142c21466992d93` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:10.115408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=325db81829e946ef8142c21466992d93)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:10.117433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:10.163113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325db81829e946ef8142c21466992d93'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.180228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325db81829e946ef8142c21466992d93'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.181440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325db81829e946ef8142c21466992d93'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.192948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325db81829e946ef8142c21466992d93'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.193922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325db81829e946ef8142c21466992d93'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.209900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1253/2504] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:10 2022] DELETE /identity/v3/users/325db81829e946ef8142c21466992d93 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:10.384250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5d90e-e7ed-4520-9be9-22b12c84b6c7 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:10.396330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5d90e-e7ed-4520-9be9-22b12c84b6c7 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:10.455211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:10.456063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:10.456344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:10.456615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] PATH_INFO: `/v3/projects/2428a60e3a0243c1a250ecb046138e36` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:10.457774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2428a60e3a0243c1a250ecb046138e36)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:10.459753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:10.470689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:10.507249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2428a60e3a0243c1a250ecb046138e36', 'request_id': 'req-2668e0b1-339c-4f1a-b2e8-23a57a540758'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.513459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2428a60e3a0243c1a250ecb046138e36', 'request_id': 'req-2668e0b1-339c-4f1a-b2e8-23a57a540758'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.521090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2428a60e3a0243c1a250ecb046138e36', 'request_id': 'req-2668e0b1-339c-4f1a-b2e8-23a57a540758'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.532847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] The token cache is being invalidate because project 2428a60e3a0243c1a250ecb046138e36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:10.533137 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.533760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.535560 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1252/2505] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:10 2022] DELETE /identity/v3/projects/2428a60e3a0243c1a250ecb046138e36 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:10.539393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:10.540853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:10.553859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ad1a6-4c14-4305-9c7b-a8c1299dc800 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:10.619915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c8600-8128-4ae7-b3d5-2c9462a8c7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:10.620936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8600-8128-4ae7-b3d5-2c9462a8c7cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:10.621219 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8600-8128-4ae7-b3d5-2c9462a8c7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:10.621523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c8600-8128-4ae7-b3d5-2c9462a8c7cc admin admin] PATH_INFO: `/v3/users/77256c08ae7c42c3acc754f691d7ca91` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:10.623334 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c8600-8128-4ae7-b3d5-2c9462a8c7cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=77256c08ae7c42c3acc754f691d7ca91)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:10.625430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c8600-8128-4ae7-b3d5-2c9462a8c7cc admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:10.677081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-005c8600-8128-4ae7-b3d5-2c9462a8c7cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77256c08ae7c42c3acc754f691d7ca91'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.682132 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:10.691949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-005c8600-8128-4ae7-b3d5-2c9462a8c7cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77256c08ae7c42c3acc754f691d7ca91'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.692772 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-005c8600-8128-4ae7-b3d5-2c9462a8c7cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77256c08ae7c42c3acc754f691d7ca91'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.701465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-005c8600-8128-4ae7-b3d5-2c9462a8c7cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77256c08ae7c42c3acc754f691d7ca91'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.702104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-005c8600-8128-4ae7-b3d5-2c9462a8c7cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77256c08ae7c42c3acc754f691d7ca91'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:10.714477 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1254/2506] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:10 2022] DELETE /identity/v3/users/77256c08ae7c42c3acc754f691d7ca91 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:10.898519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:10.900425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:10.914447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668e0b1-339c-4f1a-b2e8-23a57a540758 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:10.973797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:10.973797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:10.973797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:10.973797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] PATH_INFO: `/v3/projects/80636d08a25f49ce85206d054da815b5` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:10.975051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80636d08a25f49ce85206d054da815b5)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:10.976760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:11.024099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80636d08a25f49ce85206d054da815b5', 'request_id': 'req-a97a6065-71a0-4a50-89f0-2826c090aa09'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:11.030823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80636d08a25f49ce85206d054da815b5', 'request_id': 'req-a97a6065-71a0-4a50-89f0-2826c090aa09'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:11.040380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80636d08a25f49ce85206d054da815b5', 'request_id': 'req-a97a6065-71a0-4a50-89f0-2826c090aa09'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:11.052774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] The token cache is being invalidate because project 80636d08a25f49ce85206d054da815b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:11.053260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:11.054181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-a97a6065-71a0-4a50-89f0-2826c090aa09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:11.056694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1253/2507] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:10 2022] DELETE /identity/v3/projects/80636d08a25f49ce85206d054da815b5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:11.063836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef6dc8-a03d-4fec-8e4d-01b229da65ac None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.064322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef6dc8-a03d-4fec-8e4d-01b229da65ac None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.064453 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef6dc8-a03d-4fec-8e4d-01b229da65ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.082489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-06ef6dc8-a03d-4fec-8e4d-01b229da65ac None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:11.083702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-06ef6dc8-a03d-4fec-8e4d-01b229da65ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:11.143475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1255/2508] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:11.151683 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7e939-e0f3-4f91-a8f5-148299f6e832 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.151990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7e939-e0f3-4f91-a8f5-148299f6e832 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.152269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7e939-e0f3-4f91-a8f5-148299f6e832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.171852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-2fb7e939-e0f3-4f91-a8f5-148299f6e832 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:11.173981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-2fb7e939-e0f3-4f91-a8f5-148299f6e832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:11.182672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1254/2509] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:11.187704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef6dc8-a03d-4fec-8e4d-01b229da65ac None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:11.197563 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef6dc8-a03d-4fec-8e4d-01b229da65ac None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:11.202228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1497c-e8a7-434b-8ae7-c6babcadc011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:11.202901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1497c-e8a7-434b-8ae7-c6babcadc011 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.203178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1497c-e8a7-434b-8ae7-c6babcadc011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.203447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1497c-e8a7-434b-8ae7-c6babcadc011 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.204161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1497c-e8a7-434b-8ae7-c6babcadc011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:11.204463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1497c-e8a7-434b-8ae7-c6babcadc011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:11.206059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1497c-e8a7-434b-8ae7-c6babcadc011 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:11.218055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1256/2510] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:34:11.226629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7e939-e0f3-4f91-a8f5-148299f6e832 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:11.232211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7e939-e0f3-4f91-a8f5-148299f6e832 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:11.236827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0e197-0512-4901-b0fc-a5d007c2c2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:11.237478 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0e197-0512-4901-b0fc-a5d007c2c2ea admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.237760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0e197-0512-4901-b0fc-a5d007c2c2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.238036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0e197-0512-4901-b0fc-a5d007c2c2ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.238762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0e197-0512-4901-b0fc-a5d007c2c2ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:11.240703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0e197-0512-4901-b0fc-a5d007c2c2ea admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:11.265214 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:11.272944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1255/2511] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:11 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:11.277795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1497c-e8a7-434b-8ae7-c6babcadc011 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:11.288162 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1497c-e8a7-434b-8ae7-c6babcadc011 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:11.344472 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f60af-bd1e-4a11-be54-e9ed1752c30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:11.345153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f60af-bd1e-4a11-be54-e9ed1752c30e admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.345440 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f60af-bd1e-4a11-be54-e9ed1752c30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.345715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6f60af-bd1e-4a11-be54-e9ed1752c30e admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.346415 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f60af-bd1e-4a11-be54-e9ed1752c30e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:11.348414 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6f60af-bd1e-4a11-be54-e9ed1752c30e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:11.369496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1257/2512] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:11 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:11.373602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0e197-0512-4901-b0fc-a5d007c2c2ea admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:11.384220 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0e197-0512-4901-b0fc-a5d007c2c2ea admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:11.389634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1602fce-3663-428d-97cf-fe651ed4cf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:11.390353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602fce-3663-428d-97cf-fe651ed4cf7a admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.390672 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602fce-3663-428d-97cf-fe651ed4cf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.390948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1602fce-3663-428d-97cf-fe651ed4cf7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.391681 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1602fce-3663-428d-97cf-fe651ed4cf7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:11.393337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1602fce-3663-428d-97cf-fe651ed4cf7a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:11.408304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1256/2513] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:11.412337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f60af-bd1e-4a11-be54-e9ed1752c30e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:11.423403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6f60af-bd1e-4a11-be54-e9ed1752c30e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:11.428024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db6258-b3b4-4c76-9e42-26cb800416a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:11.428681 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db6258-b3b4-4c76-9e42-26cb800416a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.428969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db6258-b3b4-4c76-9e42-26cb800416a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.429250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db6258-b3b4-4c76-9e42-26cb800416a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.429979 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db6258-b3b4-4c76-9e42-26cb800416a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:11.431723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db6258-b3b4-4c76-9e42-26cb800416a0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:11.442020 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1258/2514] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:11.446024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1602fce-3663-428d-97cf-fe651ed4cf7a admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:11.460132 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1602fce-3663-428d-97cf-fe651ed4cf7a admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:11.464988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43050fe4-0929-4b02-bdfe-6add3791918d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:11.466399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43050fe4-0929-4b02-bdfe-6add3791918d admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.466717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43050fe4-0929-4b02-bdfe-6add3791918d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.466993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43050fe4-0929-4b02-bdfe-6add3791918d admin admin] PATH_INFO: `/v3/projects/df73c528ab77452e96229ab15e19e657/users/84f301ae649840658e417aad0904f46b/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.482445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43050fe4-0929-4b02-bdfe-6add3791918d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df73c528ab77452e96229ab15e19e657, user_id=84f301ae649840658e417aad0904f46b, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:11.485129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43050fe4-0929-4b02-bdfe-6add3791918d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:11.497956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1257/2515] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:11 2022] PUT /identity/v3/projects/df73c528ab77452e96229ab15e19e657/users/84f301ae649840658e417aad0904f46b/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:11.508386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74799997-4139-41c8-bc56-e88348ae8b40 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.508678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74799997-4139-41c8-bc56-e88348ae8b40 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.508957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74799997-4139-41c8-bc56-e88348ae8b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.537583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-74799997-4139-41c8-bc56-e88348ae8b40 None None] MFA Rules not processed for user `84f301ae649840658e417aad0904f46b`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:11.538844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-74799997-4139-41c8-bc56-e88348ae8b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:11.562912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:11.604725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1259/2516] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:11.610004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43050fe4-0929-4b02-bdfe-6add3791918d admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:11.684711 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43050fe4-0929-4b02-bdfe-6add3791918d admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:11.691296 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba87bf2-6440-4c01-8391-7812bebbdd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:11.692104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba87bf2-6440-4c01-8391-7812bebbdd0c admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.692304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba87bf2-6440-4c01-8391-7812bebbdd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.692595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba87bf2-6440-4c01-8391-7812bebbdd0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.693272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba87bf2-6440-4c01-8391-7812bebbdd0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:11.695410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba87bf2-6440-4c01-8391-7812bebbdd0c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:11.739198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1258/2517] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:11 2022] POST /identity/v3/projects => generated 387 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:11.743158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74799997-4139-41c8-bc56-e88348ae8b40 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:11.755351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74799997-4139-41c8-bc56-e88348ae8b40 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:11.824735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512195e-a2c8-48bc-b238-4dd17b8d8306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:11.826244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512195e-a2c8-48bc-b238-4dd17b8d8306 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.826583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512195e-a2c8-48bc-b238-4dd17b8d8306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.827628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6512195e-a2c8-48bc-b238-4dd17b8d8306 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.828711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512195e-a2c8-48bc-b238-4dd17b8d8306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:11.832780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6512195e-a2c8-48bc-b238-4dd17b8d8306 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:11.857455 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1260/2518] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:11 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:11.862353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba87bf2-6440-4c01-8391-7812bebbdd0c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:11.874148 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba87bf2-6440-4c01-8391-7812bebbdd0c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:11.879203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadbdca-d925-4d73-9a2a-0a92a30b23e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:11.880242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadbdca-d925-4d73-9a2a-0a92a30b23e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.880469 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadbdca-d925-4d73-9a2a-0a92a30b23e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.880961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadbdca-d925-4d73-9a2a-0a92a30b23e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.881639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadbdca-d925-4d73-9a2a-0a92a30b23e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:11.883578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadbdca-d925-4d73-9a2a-0a92a30b23e5 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:11.896447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1259/2519] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:11.901127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512195e-a2c8-48bc-b238-4dd17b8d8306 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:11.912144 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6512195e-a2c8-48bc-b238-4dd17b8d8306 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:11.917966 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54522d25-6a17-40b6-bf29-cf402e89a6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:11.918961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54522d25-6a17-40b6-bf29-cf402e89a6bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:11.919305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54522d25-6a17-40b6-bf29-cf402e89a6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:11.919645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54522d25-6a17-40b6-bf29-cf402e89a6bd admin admin] PATH_INFO: `/v3/projects/f08184e4ea4c4866aee05cc6c5f5bff0/users/1bfb1bd84b004eaeb83dbe3bbac9c9d1/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:11.936103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54522d25-6a17-40b6-bf29-cf402e89a6bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f08184e4ea4c4866aee05cc6c5f5bff0, user_id=1bfb1bd84b004eaeb83dbe3bbac9c9d1, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:11.939032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54522d25-6a17-40b6-bf29-cf402e89a6bd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:11.962636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1261/2520] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:11 2022] PUT /identity/v3/projects/f08184e4ea4c4866aee05cc6c5f5bff0/users/1bfb1bd84b004eaeb83dbe3bbac9c9d1/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:11.965508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadbdca-d925-4d73-9a2a-0a92a30b23e5 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:11.987707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadbdca-d925-4d73-9a2a-0a92a30b23e5 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:12.049448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0abf13-5723-4290-86da-89b8a6a849a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:12.050177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0abf13-5723-4290-86da-89b8a6a849a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:12.050463 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0abf13-5723-4290-86da-89b8a6a849a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:12.050768 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0abf13-5723-4290-86da-89b8a6a849a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:12.051518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0abf13-5723-4290-86da-89b8a6a849a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:12.053174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0abf13-5723-4290-86da-89b8a6a849a8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:12.067756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1260/2521] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:11 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:12.072948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54522d25-6a17-40b6-bf29-cf402e89a6bd admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:12.095365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54522d25-6a17-40b6-bf29-cf402e89a6bd admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:12.107645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3877f4-cbd8-478a-b32b-28a0cf6603a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:12.107645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3877f4-cbd8-478a-b32b-28a0cf6603a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:12.107645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3877f4-cbd8-478a-b32b-28a0cf6603a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:12.107645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3877f4-cbd8-478a-b32b-28a0cf6603a3 admin admin] PATH_INFO: `/v3/projects/f08184e4ea4c4866aee05cc6c5f5bff0/users/1bfb1bd84b004eaeb83dbe3bbac9c9d1/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:12.111179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3877f4-cbd8-478a-b32b-28a0cf6603a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f08184e4ea4c4866aee05cc6c5f5bff0, user_id=1bfb1bd84b004eaeb83dbe3bbac9c9d1, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:12.114057 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3877f4-cbd8-478a-b32b-28a0cf6603a3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:12.131930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1262/2522] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:12 2022] PUT /identity/v3/projects/f08184e4ea4c4866aee05cc6c5f5bff0/users/1bfb1bd84b004eaeb83dbe3bbac9c9d1/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:12.149784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb9dba-c984-4092-8b81-64fb8c724107 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:12.150269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb9dba-c984-4092-8b81-64fb8c724107 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:12.150746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb9dba-c984-4092-8b81-64fb8c724107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:12.182091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-3beb9dba-c984-4092-8b81-64fb8c724107 None None] MFA Rules not processed for user `1bfb1bd84b004eaeb83dbe3bbac9c9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:12.183735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3beb9dba-c984-4092-8b81-64fb8c724107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:12.220109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:12.283850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1261/2523] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:12.296259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3877f4-cbd8-478a-b32b-28a0cf6603a3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:12.298033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-4e3877f4-cbd8-478a-b32b-28a0cf6603a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:12.362053 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3877f4-cbd8-478a-b32b-28a0cf6603a3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:12.366706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5766a490-4210-4c9f-9967-02f666a0d1b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:12.367372 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766a490-4210-4c9f-9967-02f666a0d1b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:12.367662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766a490-4210-4c9f-9967-02f666a0d1b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:12.367937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5766a490-4210-4c9f-9967-02f666a0d1b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:12.377564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766a490-4210-4c9f-9967-02f666a0d1b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:12.379154 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5766a490-4210-4c9f-9967-02f666a0d1b1 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:12.390643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1263/2524] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:12.456818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb9dba-c984-4092-8b81-64fb8c724107 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:12.529410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb9dba-c984-4092-8b81-64fb8c724107 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:12.534342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:12.535245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:12.535524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:12.535794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] PATH_INFO: `/v3/users/84f301ae649840658e417aad0904f46b` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:12.537058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f301ae649840658e417aad0904f46b)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:12.539135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:12.586083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f301ae649840658e417aad0904f46b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:12.599071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f301ae649840658e417aad0904f46b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:12.599733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f301ae649840658e417aad0904f46b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:12.606660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f301ae649840658e417aad0904f46b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:12.607288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f301ae649840658e417aad0904f46b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:12.619821 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1262/2525] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:12 2022] DELETE /identity/v3/users/84f301ae649840658e417aad0904f46b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:12.624899 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287f279-5bf8-4d51-986f-67aa85bf68e9 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:12.625208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287f279-5bf8-4d51-986f-67aa85bf68e9 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:12.625515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287f279-5bf8-4d51-986f-67aa85bf68e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:12.643950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c287f279-5bf8-4d51-986f-67aa85bf68e9 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:12.645147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c287f279-5bf8-4d51-986f-67aa85bf68e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:12.663636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:12.713826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1264/2526] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:12.723998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:12.725415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:12.769691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:12.785771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0481b5d-be92-431c-9a78-771f74a43825 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:12.790394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80625a73-845a-450e-957a-55fdca1a5daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:12.791117 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80625a73-845a-450e-957a-55fdca1a5daf service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:12.791619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80625a73-845a-450e-957a-55fdca1a5daf service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:12.791924 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80625a73-845a-450e-957a-55fdca1a5daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:12.802182 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80625a73-845a-450e-957a-55fdca1a5daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:12.804378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80625a73-845a-450e-957a-55fdca1a5daf service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:12.816291 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1263/2527] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:13.091783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287f279-5bf8-4d51-986f-67aa85bf68e9 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:13.094092 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c287f279-5bf8-4d51-986f-67aa85bf68e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:13.108585 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287f279-5bf8-4d51-986f-67aa85bf68e9 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:13.114029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:13.115366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:13.115585 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:13.115927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] PATH_INFO: `/v3/projects/df73c528ab77452e96229ab15e19e657` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:13.117092 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df73c528ab77452e96229ab15e19e657)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:13.119071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:13.175185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df73c528ab77452e96229ab15e19e657', 'request_id': 'req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.181543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df73c528ab77452e96229ab15e19e657', 'request_id': 'req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.189630 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df73c528ab77452e96229ab15e19e657', 'request_id': 'req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.203174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] The token cache is being invalidate because project df73c528ab77452e96229ab15e19e657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:13.203464 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.204111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.205667 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1265/2528] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:13 2022] DELETE /identity/v3/projects/df73c528ab77452e96229ab15e19e657 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:13.209796 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80625a73-845a-450e-957a-55fdca1a5daf service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:13.219851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80625a73-845a-450e-957a-55fdca1a5daf service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:13.281759 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393f3e1-4f09-414b-bc41-2fd2dedcea5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:13.282659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f3e1-4f09-414b-bc41-2fd2dedcea5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:13.282943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f3e1-4f09-414b-bc41-2fd2dedcea5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:13.283218 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f3e1-4f09-414b-bc41-2fd2dedcea5c admin admin] PATH_INFO: `/v3/users/1bfb1bd84b004eaeb83dbe3bbac9c9d1` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:13.284636 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393f3e1-4f09-414b-bc41-2fd2dedcea5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bfb1bd84b004eaeb83dbe3bbac9c9d1)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:13.286718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393f3e1-4f09-414b-bc41-2fd2dedcea5c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:13.338883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f393f3e1-4f09-414b-bc41-2fd2dedcea5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bfb1bd84b004eaeb83dbe3bbac9c9d1'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.351176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f393f3e1-4f09-414b-bc41-2fd2dedcea5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bfb1bd84b004eaeb83dbe3bbac9c9d1'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.351871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f393f3e1-4f09-414b-bc41-2fd2dedcea5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bfb1bd84b004eaeb83dbe3bbac9c9d1'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.360431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f393f3e1-4f09-414b-bc41-2fd2dedcea5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bfb1bd84b004eaeb83dbe3bbac9c9d1'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.361098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-f393f3e1-4f09-414b-bc41-2fd2dedcea5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bfb1bd84b004eaeb83dbe3bbac9c9d1'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.376169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1264/2529] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:13 2022] DELETE /identity/v3/users/1bfb1bd84b004eaeb83dbe3bbac9c9d1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:13.693215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:13.694890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:13.708218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a900fe-a4bf-4cdf-9f8d-f4de2b8e2c21 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:13.768109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:13.769017 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:13.769302 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:13.769922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] PATH_INFO: `/v3/projects/f08184e4ea4c4866aee05cc6c5f5bff0` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:13.771254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f08184e4ea4c4866aee05cc6c5f5bff0)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:13.773343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:13.828174 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:13.831830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f08184e4ea4c4866aee05cc6c5f5bff0', 'request_id': 'req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.838446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f08184e4ea4c4866aee05cc6c5f5bff0', 'request_id': 'req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.846565 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f08184e4ea4c4866aee05cc6c5f5bff0', 'request_id': 'req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.858552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] The token cache is being invalidate because project f08184e4ea4c4866aee05cc6c5f5bff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:13.858830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.859498 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-38e5cf5a-7cdb-4111-ace4-86e54a475a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:13.861065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1266/2530] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:13 2022] DELETE /identity/v3/projects/f08184e4ea4c4866aee05cc6c5f5bff0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:13.866551 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0233bc-e1a5-4306-b463-9820b0d2b4d1 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:13.867123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0233bc-e1a5-4306-b463-9820b0d2b4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:13.867226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0233bc-e1a5-4306-b463-9820b0d2b4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:13.887320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0c0233bc-e1a5-4306-b463-9820b0d2b4d1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:13.888638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0c0233bc-e1a5-4306-b463-9820b0d2b4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:13.967289 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1265/2531] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:13.975041 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a69cab-b691-44b1-895c-f65ef65a93c7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:13.975328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a69cab-b691-44b1-895c-f65ef65a93c7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:13.975606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a69cab-b691-44b1-895c-f65ef65a93c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:13.994094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-65a69cab-b691-44b1-895c-f65ef65a93c7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:13.995364 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-65a69cab-b691-44b1-895c-f65ef65a93c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:14.004218 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1267/2532] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:14.008648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0233bc-e1a5-4306-b463-9820b0d2b4d1 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.014256 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:14.020942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0233bc-e1a5-4306-b463-9820b0d2b4d1 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.027205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daed8be-db86-436a-9bcb-1f948a6bce90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.028039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daed8be-db86-436a-9bcb-1f948a6bce90 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.029053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daed8be-db86-436a-9bcb-1f948a6bce90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.029334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daed8be-db86-436a-9bcb-1f948a6bce90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.030248 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daed8be-db86-436a-9bcb-1f948a6bce90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:14.030693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daed8be-db86-436a-9bcb-1f948a6bce90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.032784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daed8be-db86-436a-9bcb-1f948a6bce90 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.047246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1266/2533] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:34:14.051539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a69cab-b691-44b1-895c-f65ef65a93c7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.061601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a69cab-b691-44b1-895c-f65ef65a93c7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.066725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d3dab-e34a-475a-885b-d2b6855ad775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.067393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d3dab-e34a-475a-885b-d2b6855ad775 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.067663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d3dab-e34a-475a-885b-d2b6855ad775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.067937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d3dab-e34a-475a-885b-d2b6855ad775 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.068662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d3dab-e34a-475a-885b-d2b6855ad775 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.070639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d3dab-e34a-475a-885b-d2b6855ad775 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.101809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1268/2534] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:14 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:14.106319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daed8be-db86-436a-9bcb-1f948a6bce90 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.116935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daed8be-db86-436a-9bcb-1f948a6bce90 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.179836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8af890-adbc-4da9-8840-cb07a28383e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.180535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8af890-adbc-4da9-8840-cb07a28383e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.180824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8af890-adbc-4da9-8840-cb07a28383e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.181104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8af890-adbc-4da9-8840-cb07a28383e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.181797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8af890-adbc-4da9-8840-cb07a28383e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.183808 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8af890-adbc-4da9-8840-cb07a28383e7 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.208454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1267/2535] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:14 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:14.210000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d3dab-e34a-475a-885b-d2b6855ad775 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.227111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d3dab-e34a-475a-885b-d2b6855ad775 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.232900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41280f5-bf45-4970-a576-fb92005254ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.233586 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41280f5-bf45-4970-a576-fb92005254ce admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.233864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41280f5-bf45-4970-a576-fb92005254ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.234145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41280f5-bf45-4970-a576-fb92005254ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.234976 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41280f5-bf45-4970-a576-fb92005254ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.236598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41280f5-bf45-4970-a576-fb92005254ce admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.251036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1269/2536] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:14.254878 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8af890-adbc-4da9-8840-cb07a28383e7 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.265586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8af890-adbc-4da9-8840-cb07a28383e7 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.270406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61996bf-60d7-49df-a407-660abae47cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.271122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61996bf-60d7-49df-a407-660abae47cfb admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.271407 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61996bf-60d7-49df-a407-660abae47cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.272092 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61996bf-60d7-49df-a407-660abae47cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.272418 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61996bf-60d7-49df-a407-660abae47cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.274031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61996bf-60d7-49df-a407-660abae47cfb admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.284883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1268/2537] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:14.289070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41280f5-bf45-4970-a576-fb92005254ce admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.300751 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41280f5-bf45-4970-a576-fb92005254ce admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.307616 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5ec12-223a-4882-8421-b706cb01fd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.308590 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5ec12-223a-4882-8421-b706cb01fd46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.308882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5ec12-223a-4882-8421-b706cb01fd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.309161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b5ec12-223a-4882-8421-b706cb01fd46 admin admin] PATH_INFO: `/v3/projects/34b6407300484765a75117a62b341dc6/users/17350754fd044459ba581f070e1aadbf/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.327975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5ec12-223a-4882-8421-b706cb01fd46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34b6407300484765a75117a62b341dc6, user_id=17350754fd044459ba581f070e1aadbf, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.330704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b5ec12-223a-4882-8421-b706cb01fd46 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.349586 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1270/2538] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:14 2022] PUT /identity/v3/projects/34b6407300484765a75117a62b341dc6/users/17350754fd044459ba581f070e1aadbf/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:14.360309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53594743-8091-4daa-ab59-c6d12dcc13e0 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.360612 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53594743-8091-4daa-ab59-c6d12dcc13e0 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.360881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53594743-8091-4daa-ab59-c6d12dcc13e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.399959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-53594743-8091-4daa-ab59-c6d12dcc13e0 None None] MFA Rules not processed for user `17350754fd044459ba581f070e1aadbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:14.400974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-53594743-8091-4daa-ab59-c6d12dcc13e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:14.477248 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1269/2539] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:14.482723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5ec12-223a-4882-8421-b706cb01fd46 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.551916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b5ec12-223a-4882-8421-b706cb01fd46 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.557074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb1f9e-deb2-4830-9b14-7caa90a6afa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.557734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb1f9e-deb2-4830-9b14-7caa90a6afa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.558015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb1f9e-deb2-4830-9b14-7caa90a6afa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.558289 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb1f9e-deb2-4830-9b14-7caa90a6afa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.559046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb1f9e-deb2-4830-9b14-7caa90a6afa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.561535 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb1f9e-deb2-4830-9b14-7caa90a6afa7 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.591346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1271/2540] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:14 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:14.595509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53594743-8091-4daa-ab59-c6d12dcc13e0 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.606712 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53594743-8091-4daa-ab59-c6d12dcc13e0 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.664033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:14.672145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e12aa2-74af-45c4-85a3-e4006e980a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.672831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e12aa2-74af-45c4-85a3-e4006e980a73 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.673112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e12aa2-74af-45c4-85a3-e4006e980a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.673389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e12aa2-74af-45c4-85a3-e4006e980a73 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.674086 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e12aa2-74af-45c4-85a3-e4006e980a73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.676521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e12aa2-74af-45c4-85a3-e4006e980a73 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.701840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1270/2541] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:14 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:14.703096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb1f9e-deb2-4830-9b14-7caa90a6afa7 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.713454 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb1f9e-deb2-4830-9b14-7caa90a6afa7 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.718277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8fef0-c3aa-436c-b7c8-1eebc73f0006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.719001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8fef0-c3aa-436c-b7c8-1eebc73f0006 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.719283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8fef0-c3aa-436c-b7c8-1eebc73f0006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.719561 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8fef0-c3aa-436c-b7c8-1eebc73f0006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.720323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8fef0-c3aa-436c-b7c8-1eebc73f0006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.721929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8fef0-c3aa-436c-b7c8-1eebc73f0006 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.733339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1272/2542] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:14.737545 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e12aa2-74af-45c4-85a3-e4006e980a73 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.748638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e12aa2-74af-45c4-85a3-e4006e980a73 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.753630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59e5b5-9ff2-4942-bbd9-ccf6a96e3169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.754563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e5b5-9ff2-4942-bbd9-ccf6a96e3169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.754951 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e5b5-9ff2-4942-bbd9-ccf6a96e3169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.755242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59e5b5-9ff2-4942-bbd9-ccf6a96e3169 admin admin] PATH_INFO: `/v3/projects/1353749388dd4d13838d0a2fc305489a/users/a998fea0f0244279861c542e0ad7f471/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.771540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59e5b5-9ff2-4942-bbd9-ccf6a96e3169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1353749388dd4d13838d0a2fc305489a, user_id=a998fea0f0244279861c542e0ad7f471, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.774217 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59e5b5-9ff2-4942-bbd9-ccf6a96e3169 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.787381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1271/2543] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:14 2022] PUT /identity/v3/projects/1353749388dd4d13838d0a2fc305489a/users/a998fea0f0244279861c542e0ad7f471/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:14.791616 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8fef0-c3aa-436c-b7c8-1eebc73f0006 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.805707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8fef0-c3aa-436c-b7c8-1eebc73f0006 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.864670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:14.886875 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978c573-a13f-4e83-9349-124399a64c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.888246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978c573-a13f-4e83-9349-124399a64c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.888530 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978c573-a13f-4e83-9349-124399a64c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.888808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978c573-a13f-4e83-9349-124399a64c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.889536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978c573-a13f-4e83-9349-124399a64c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.891284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978c573-a13f-4e83-9349-124399a64c71 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.901904 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1273/2544] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:14.905886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59e5b5-9ff2-4942-bbd9-ccf6a96e3169 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:14.916383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59e5b5-9ff2-4942-bbd9-ccf6a96e3169 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:14.922230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597a74b-87d0-4bed-939e-7a7347aa4900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:14.923155 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597a74b-87d0-4bed-939e-7a7347aa4900 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.923464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597a74b-87d0-4bed-939e-7a7347aa4900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.923746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597a74b-87d0-4bed-939e-7a7347aa4900 admin admin] PATH_INFO: `/v3/projects/1353749388dd4d13838d0a2fc305489a/users/a998fea0f0244279861c542e0ad7f471/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.925770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597a74b-87d0-4bed-939e-7a7347aa4900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1353749388dd4d13838d0a2fc305489a, user_id=a998fea0f0244279861c542e0ad7f471, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:14.928480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597a74b-87d0-4bed-939e-7a7347aa4900 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:14.939362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1272/2545] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:14 2022] PUT /identity/v3/projects/1353749388dd4d13838d0a2fc305489a/users/a998fea0f0244279861c542e0ad7f471/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:14.951351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab9003-55b6-4c35-a160-dc7e40e40545 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:14.951636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab9003-55b6-4c35-a160-dc7e40e40545 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:14.951918 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab9003-55b6-4c35-a160-dc7e40e40545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:14.979262 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a0ab9003-55b6-4c35-a160-dc7e40e40545 None None] MFA Rules not processed for user `a998fea0f0244279861c542e0ad7f471`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:14.980336 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a0ab9003-55b6-4c35-a160-dc7e40e40545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:15.053160 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1274/2546] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:14 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:15.065883 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597a74b-87d0-4bed-939e-7a7347aa4900 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:15.067632 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3597a74b-87d0-4bed-939e-7a7347aa4900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:15.124234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597a74b-87d0-4bed-939e-7a7347aa4900 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:15.128414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29502f80-20d0-4871-8414-8eab0ac37c15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:15.128984 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29502f80-20d0-4871-8414-8eab0ac37c15 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:15.129264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29502f80-20d0-4871-8414-8eab0ac37c15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:15.129537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29502f80-20d0-4871-8414-8eab0ac37c15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:15.139924 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29502f80-20d0-4871-8414-8eab0ac37c15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:15.141349 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29502f80-20d0-4871-8414-8eab0ac37c15 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:15.151969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1273/2547] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:15.492938 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab9003-55b6-4c35-a160-dc7e40e40545 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:15.504089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab9003-55b6-4c35-a160-dc7e40e40545 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:15.509120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab21327-60b6-4454-8707-09e2c9a4bc32 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a998fea0f0244279861c542e0ad7f471', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1353749388dd4d13838d0a2fc305489a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:15.510315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21327-60b6-4454-8707-09e2c9a4bc32 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:15.510555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21327-60b6-4454-8707-09e2c9a4bc32 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:15.510871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab21327-60b6-4454-8707-09e2c9a4bc32 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:15.511652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab21327-60b6-4454-8707-09e2c9a4bc32 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:15.511963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab21327-60b6-4454-8707-09e2c9a4bc32 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:15.513614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab21327-60b6-4454-8707-09e2c9a4bc32 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:15.530568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1275/2548] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:34:15.533332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29502f80-20d0-4871-8414-8eab0ac37c15 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:15.542841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29502f80-20d0-4871-8414-8eab0ac37c15 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:15.548744 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c408a3a-d3cf-4333-a9d8-91ebc8243c11 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a998fea0f0244279861c542e0ad7f471', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1353749388dd4d13838d0a2fc305489a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:15.549460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c408a3a-d3cf-4333-a9d8-91ebc8243c11 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:15.549748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c408a3a-d3cf-4333-a9d8-91ebc8243c11 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:15.550133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c408a3a-d3cf-4333-a9d8-91ebc8243c11 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:15.550785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c408a3a-d3cf-4333-a9d8-91ebc8243c11 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:15.552738 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c408a3a-d3cf-4333-a9d8-91ebc8243c11 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:15.580394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1274/2549] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:15 2022] POST /identity/v3/projects => generated 354 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:15.584497 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab21327-60b6-4454-8707-09e2c9a4bc32 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:15.595801 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab21327-60b6-4454-8707-09e2c9a4bc32 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:15.620107 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c408a3a-d3cf-4333-a9d8-91ebc8243c11 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:15.622887 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-4c408a3a-d3cf-4333-a9d8-91ebc8243c11 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:15.663326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6b877-234a-430d-8a2a-9dbb7e39c512 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a998fea0f0244279861c542e0ad7f471', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1353749388dd4d13838d0a2fc305489a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:15.664037 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6b877-234a-430d-8a2a-9dbb7e39c512 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:15.664327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6b877-234a-430d-8a2a-9dbb7e39c512 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:15.664611 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6b877-234a-430d-8a2a-9dbb7e39c512 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:15.665378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6b877-234a-430d-8a2a-9dbb7e39c512 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:15.665690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6b877-234a-430d-8a2a-9dbb7e39c512 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:15.667534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6b877-234a-430d-8a2a-9dbb7e39c512 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:15.681815 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1276/2550] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:34:15.705163 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c408a3a-d3cf-4333-a9d8-91ebc8243c11 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:15.710131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd8581-29b1-4893-9620-1a836e74531d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:15.711055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd8581-29b1-4893-9620-1a836e74531d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:15.711460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd8581-29b1-4893-9620-1a836e74531d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:15.711774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd8581-29b1-4893-9620-1a836e74531d admin admin] PATH_INFO: `/v3/users/c182527fdae74c84b4c4db6f40c4904b` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:15.713026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd8581-29b1-4893-9620-1a836e74531d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c182527fdae74c84b4c4db6f40c4904b)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:15.715054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dd8581-29b1-4893-9620-1a836e74531d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:15.723391 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6b877-234a-430d-8a2a-9dbb7e39c512 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:15.734006 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6b877-234a-430d-8a2a-9dbb7e39c512 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:15.738812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a998fea0f0244279861c542e0ad7f471', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1353749388dd4d13838d0a2fc305489a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:15.739672 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:15.739958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:15.740233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] PATH_INFO: `/v3/projects/f68f50b0f8644e35b1f7444e4ee81ecd` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:15.741424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f68f50b0f8644e35b1f7444e4ee81ecd)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:15.743519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:15.754432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:15.772734 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-a3dd8581-29b1-4893-9620-1a836e74531d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c182527fdae74c84b4c4db6f40c4904b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:15.786195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-a3dd8581-29b1-4893-9620-1a836e74531d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c182527fdae74c84b4c4db6f40c4904b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:15.786914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-a3dd8581-29b1-4893-9620-1a836e74531d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c182527fdae74c84b4c4db6f40c4904b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:15.788267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:15.795821 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-a3dd8581-29b1-4893-9620-1a836e74531d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c182527fdae74c84b4c4db6f40c4904b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:15.796420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f68f50b0f8644e35b1f7444e4ee81ecd', 'request_id': 'req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:15.796553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-a3dd8581-29b1-4893-9620-1a836e74531d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c182527fdae74c84b4c4db6f40c4904b'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:15.803229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f68f50b0f8644e35b1f7444e4ee81ecd', 'request_id': 'req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:15.809001 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1275/2551] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:15 2022] DELETE /identity/v3/users/c182527fdae74c84b4c4db6f40c4904b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:15.813143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f68f50b0f8644e35b1f7444e4ee81ecd', 'request_id': 'req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:15.814733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b77cc-7f6d-4a07-a02d-6978103a6225 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:15.815020 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b77cc-7f6d-4a07-a02d-6978103a6225 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:15.815294 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b77cc-7f6d-4a07-a02d-6978103a6225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:15.830565 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] The token cache is being invalidate because project f68f50b0f8644e35b1f7444e4ee81ecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:15.830834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:15.831558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:15.832237 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-017b77cc-7f6d-4a07-a02d-6978103a6225 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:15.833138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1277/2552] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:15 2022] DELETE /identity/v3/projects/f68f50b0f8644e35b1f7444e4ee81ecd => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:15.835144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-017b77cc-7f6d-4a07-a02d-6978103a6225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:15.904605 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1276/2553] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:15.915227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:15.916886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:15.992026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61edd9-8673-403d-9f32-6ca4a6ad75c0 tempest-VolumeQuotaClassesTest-873290474 tempest-VolumeQuotaClassesTest-873290474-project-admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:16.045183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b728239-05ce-46e1-a4a6-4b21451d18f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:16.045841 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b728239-05ce-46e1-a4a6-4b21451d18f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:16.046126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b728239-05ce-46e1-a4a6-4b21451d18f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:16.046433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b728239-05ce-46e1-a4a6-4b21451d18f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:16.052629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b77cc-7f6d-4a07-a02d-6978103a6225 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:16.054662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-017b77cc-7f6d-4a07-a02d-6978103a6225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:16.056813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b728239-05ce-46e1-a4a6-4b21451d18f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:16.058277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b728239-05ce-46e1-a4a6-4b21451d18f8 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:16.068095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b77cc-7f6d-4a07-a02d-6978103a6225 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:16.069045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1278/2554] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:16.073443 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:16.074440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:16.074771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:16.075049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] PATH_INFO: `/v3/users/17350754fd044459ba581f070e1aadbf` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:16.085419 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17350754fd044459ba581f070e1aadbf)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:16.085419 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:16.132520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17350754fd044459ba581f070e1aadbf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.149538 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17350754fd044459ba581f070e1aadbf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.150352 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17350754fd044459ba581f070e1aadbf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.158598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17350754fd044459ba581f070e1aadbf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.159318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17350754fd044459ba581f070e1aadbf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.176614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1277/2555] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:16 2022] DELETE /identity/v3/users/17350754fd044459ba581f070e1aadbf => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:16.181673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ecf36-9ae0-4e8b-b0d8-11369a484bd5 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:16.182186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ecf36-9ae0-4e8b-b0d8-11369a484bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:16.182842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ecf36-9ae0-4e8b-b0d8-11369a484bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:16.202749 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b84ecf36-9ae0-4e8b-b0d8-11369a484bd5 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:16.204067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b84ecf36-9ae0-4e8b-b0d8-11369a484bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:16.285562 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1279/2556] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:16.295336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:16.296758 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:16.325044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:16.361213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cfe5a-c10a-42d8-ba86-0e1241f297c2 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:16.364488 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ecf36-9ae0-4e8b-b0d8-11369a484bd5 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:16.365971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efc2cc-0ddf-4779-a548-a3f9ac19d362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:16.366225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b84ecf36-9ae0-4e8b-b0d8-11369a484bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:16.366824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efc2cc-0ddf-4779-a548-a3f9ac19d362 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:16.367104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efc2cc-0ddf-4779-a548-a3f9ac19d362 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:16.367382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7efc2cc-0ddf-4779-a548-a3f9ac19d362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:16.378943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efc2cc-0ddf-4779-a548-a3f9ac19d362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:16.380599 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7efc2cc-0ddf-4779-a548-a3f9ac19d362 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:16.387141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ecf36-9ae0-4e8b-b0d8-11369a484bd5 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:16.397390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:16.397390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:16.397390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:16.397390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] PATH_INFO: `/v3/projects/0b3962c8fc204046af26c5f91d3ded13` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:16.398225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b3962c8fc204046af26c5f91d3ded13)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:16.399625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:16.402632 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1278/2557] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:16.434271 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:16.472448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b3962c8fc204046af26c5f91d3ded13', 'request_id': 'req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.482109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b3962c8fc204046af26c5f91d3ded13', 'request_id': 'req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.492242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b3962c8fc204046af26c5f91d3ded13', 'request_id': 'req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.504597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] The token cache is being invalidate because project 0b3962c8fc204046af26c5f91d3ded13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:16.505652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.505652 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.507164 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1280/2558] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:16 2022] DELETE /identity/v3/projects/0b3962c8fc204046af26c5f91d3ded13 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:16.514909 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efc2cc-0ddf-4779-a548-a3f9ac19d362 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:16.532399 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7efc2cc-0ddf-4779-a548-a3f9ac19d362 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:16.628146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:16.629125 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:16.629510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:16.629902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] PATH_INFO: `/v3/users/d0772f709b1940f9a5991b7cc41547df` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:16.631610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0772f709b1940f9a5991b7cc41547df)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:16.633886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:16.698665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0772f709b1940f9a5991b7cc41547df'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.711882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0772f709b1940f9a5991b7cc41547df'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.712620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0772f709b1940f9a5991b7cc41547df'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.720118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0772f709b1940f9a5991b7cc41547df'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.720847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0772f709b1940f9a5991b7cc41547df'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.736822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1279/2559] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:16 2022] DELETE /identity/v3/users/d0772f709b1940f9a5991b7cc41547df => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:16.748073 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:16.749934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:16.772613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2cbb6-9227-4e4d-9cf4-878bcaaf8e6e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:16.868544 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:16.869449 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:16.869730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:16.870046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] PATH_INFO: `/v3/projects/34b6407300484765a75117a62b341dc6` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:16.871350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] RBAC: Authorizing `identity:delete_project(project_id=34b6407300484765a75117a62b341dc6)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:16.873840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:16.926608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34b6407300484765a75117a62b341dc6', 'request_id': 'req-ce11bdb5-2391-46b8-b898-54c596a7663b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.932600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34b6407300484765a75117a62b341dc6', 'request_id': 'req-ce11bdb5-2391-46b8-b898-54c596a7663b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.941810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34b6407300484765a75117a62b341dc6', 'request_id': 'req-ce11bdb5-2391-46b8-b898-54c596a7663b'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.954994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] The token cache is being invalidate because project 34b6407300484765a75117a62b341dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:16.955201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.956011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:16.957632 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1281/2560] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:16 2022] DELETE /identity/v3/projects/34b6407300484765a75117a62b341dc6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:16.961591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:16.963206 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:16.977036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd37d42-e208-4c59-a212-c0582bf1c5e9 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:17.039208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:17.066825 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:17.067832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.068830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.069181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] PATH_INFO: `/v3/users/a998fea0f0244279861c542e0ad7f471` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.070907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] RBAC: Authorizing `identity:delete_user(user_id=a998fea0f0244279861c542e0ad7f471)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:17.073140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:17.132074 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a998fea0f0244279861c542e0ad7f471'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.145922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a998fea0f0244279861c542e0ad7f471'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.146694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a998fea0f0244279861c542e0ad7f471'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.155414 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a998fea0f0244279861c542e0ad7f471'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.156107 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a998fea0f0244279861c542e0ad7f471'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.170805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1280/2561] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:16 2022] DELETE /identity/v3/users/a998fea0f0244279861c542e0ad7f471 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:17.311620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:17.313043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:17.336384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11bdb5-2391-46b8-b898-54c596a7663b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:17.386612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:17.409065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:17.410045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.410419 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.410761 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] PATH_INFO: `/v3/projects/e518b4f271f647409fb12d891c4579a7` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.412125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e518b4f271f647409fb12d891c4579a7)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:17.414130 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:17.470070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e518b4f271f647409fb12d891c4579a7', 'request_id': 'req-044d16b5-59d0-4af2-a054-a969978a32a5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.474625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:17.476124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:17.481543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e518b4f271f647409fb12d891c4579a7', 'request_id': 'req-044d16b5-59d0-4af2-a054-a969978a32a5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.491706 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e518b4f271f647409fb12d891c4579a7', 'request_id': 'req-044d16b5-59d0-4af2-a054-a969978a32a5'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.500662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11071f-a248-4d71-badd-9a10462736af admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:17.507101 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] The token cache is being invalidate because project e518b4f271f647409fb12d891c4579a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:17.507902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.508051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-044d16b5-59d0-4af2-a054-a969978a32a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.510577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1282/2562] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:17 2022] DELETE /identity/v3/projects/e518b4f271f647409fb12d891c4579a7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:17.516495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a597852-0e8a-4b2a-bb8d-6a2d5a1ba525 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.516799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a597852-0e8a-4b2a-bb8d-6a2d5a1ba525 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.517050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a597852-0e8a-4b2a-bb8d-6a2d5a1ba525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.537256 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-1a597852-0e8a-4b2a-bb8d-6a2d5a1ba525 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:17.538936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1a597852-0e8a-4b2a-bb8d-6a2d5a1ba525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:17.613181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:17.614809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.615229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.615648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] PATH_INFO: `/v3/projects/1353749388dd4d13838d0a2fc305489a` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.617484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1353749388dd4d13838d0a2fc305489a)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:17.619815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:17.624705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1283/2563] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:17.633087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3cd5a-1e58-40d0-8f9c-ccbced43904d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.633377 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3cd5a-1e58-40d0-8f9c-ccbced43904d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.633669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3cd5a-1e58-40d0-8f9c-ccbced43904d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.655176 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-67f3cd5a-1e58-40d0-8f9c-ccbced43904d None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:17.656552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-67f3cd5a-1e58-40d0-8f9c-ccbced43904d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:17.665714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1284/2564] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:17.674703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3cd5a-1e58-40d0-8f9c-ccbced43904d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:17.680285 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3cd5a-1e58-40d0-8f9c-ccbced43904d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:17.695139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1353749388dd4d13838d0a2fc305489a', 'request_id': 'req-5065ddca-a6f0-45ce-93f9-d87a0054bf32'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.701123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1353749388dd4d13838d0a2fc305489a', 'request_id': 'req-5065ddca-a6f0-45ce-93f9-d87a0054bf32'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.709345 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1353749388dd4d13838d0a2fc305489a', 'request_id': 'req-5065ddca-a6f0-45ce-93f9-d87a0054bf32'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.710877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:17.726762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] The token cache is being invalidate because project 1353749388dd4d13838d0a2fc305489a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:17.727766 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.728810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5065ddca-a6f0-45ce-93f9-d87a0054bf32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:17.730920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1281/2565] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:17 2022] DELETE /identity/v3/projects/1353749388dd4d13838d0a2fc305489a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:17.736396 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6971bf0-6224-4a07-a13d-4605702a1b5b None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.736719 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6971bf0-6224-4a07-a13d-4605702a1b5b None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.737219 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6971bf0-6224-4a07-a13d-4605702a1b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.755822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb036a6-3d84-4295-9fb4-797ed585632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:17.756698 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb036a6-3d84-4295-9fb4-797ed585632e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.757300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb036a6-3d84-4295-9fb4-797ed585632e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.757801 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb036a6-3d84-4295-9fb4-797ed585632e admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.758734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb036a6-3d84-4295-9fb4-797ed585632e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:17.759222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb036a6-3d84-4295-9fb4-797ed585632e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:17.760615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-f6971bf0-6224-4a07-a13d-4605702a1b5b None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:17.761829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb036a6-3d84-4295-9fb4-797ed585632e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:17.762228 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f6971bf0-6224-4a07-a13d-4605702a1b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:17.772051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1282/2566] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:17.774830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:17.780692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fd7a1-feef-4911-9528-0c34dcca9871 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.781170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fd7a1-feef-4911-9528-0c34dcca9871 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.781741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fd7a1-feef-4911-9528-0c34dcca9871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.784922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1285/2567] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:34:17.789727 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb036a6-3d84-4295-9fb4-797ed585632e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:17.791941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ffb036a6-3d84-4295-9fb4-797ed585632e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:17.805548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-f72fd7a1-feef-4911-9528-0c34dcca9871 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:17.806594 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb036a6-3d84-4295-9fb4-797ed585632e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:17.807872 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f72fd7a1-feef-4911-9528-0c34dcca9871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:17.813110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84121d21-48f1-450b-92bd-42deb7c5491d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:17.813785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84121d21-48f1-450b-92bd-42deb7c5491d admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.814074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84121d21-48f1-450b-92bd-42deb7c5491d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.814351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84121d21-48f1-450b-92bd-42deb7c5491d admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.815104 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84121d21-48f1-450b-92bd-42deb7c5491d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:17.817046 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84121d21-48f1-450b-92bd-42deb7c5491d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:17.817229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1283/2568] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:17.821617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fd7a1-feef-4911-9528-0c34dcca9871 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:17.831968 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fd7a1-feef-4911-9528-0c34dcca9871 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:17.837622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcef515-b321-4573-9c80-40f4357af9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:17.838413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcef515-b321-4573-9c80-40f4357af9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.838786 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcef515-b321-4573-9c80-40f4357af9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.839102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcef515-b321-4573-9c80-40f4357af9b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.839802 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcef515-b321-4573-9c80-40f4357af9b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:17.840112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcef515-b321-4573-9c80-40f4357af9b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:17.841845 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcef515-b321-4573-9c80-40f4357af9b6 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:17.850503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1286/2569] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:17 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:17.853748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84121d21-48f1-450b-92bd-42deb7c5491d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:17.860249 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1284/2570] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:34:17.865284 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcef515-b321-4573-9c80-40f4357af9b6 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:17.877030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcef515-b321-4573-9c80-40f4357af9b6 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:17.923742 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84121d21-48f1-450b-92bd-42deb7c5491d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:17.928572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c57d4-50c5-4872-8d26-0c12e1abd71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:17.929250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c57d4-50c5-4872-8d26-0c12e1abd71d admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.929529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c57d4-50c5-4872-8d26-0c12e1abd71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.929805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c57d4-50c5-4872-8d26-0c12e1abd71d admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.930548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c57d4-50c5-4872-8d26-0c12e1abd71d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:17.932506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c57d4-50c5-4872-8d26-0c12e1abd71d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:17.937816 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719c3c5-0326-4d61-aaa3-6990cb2dae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:17.938513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719c3c5-0326-4d61-aaa3-6990cb2dae71 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:17.938871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719c3c5-0326-4d61-aaa3-6990cb2dae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:17.939162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719c3c5-0326-4d61-aaa3-6990cb2dae71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:17.939910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719c3c5-0326-4d61-aaa3-6990cb2dae71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:17.942033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a719c3c5-0326-4d61-aaa3-6990cb2dae71 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:17.954170 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1287/2571] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:17 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:17.957952 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c57d4-50c5-4872-8d26-0c12e1abd71d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:17.969462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c57d4-50c5-4872-8d26-0c12e1abd71d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:17.971101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1285/2572] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:17 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:17.975439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719c3c5-0326-4d61-aaa3-6990cb2dae71 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.033534 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f6cdd-3130-41af-9243-ec48aec91c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.034845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f6cdd-3130-41af-9243-ec48aec91c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.035150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f6cdd-3130-41af-9243-ec48aec91c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.035428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f6cdd-3130-41af-9243-ec48aec91c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.036198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f6cdd-3130-41af-9243-ec48aec91c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.037840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719c3c5-0326-4d61-aaa3-6990cb2dae71 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.037974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f6cdd-3130-41af-9243-ec48aec91c8c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.042666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc39628-fc4c-4795-92d9-b09f14eadcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.043340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc39628-fc4c-4795-92d9-b09f14eadcf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.043621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc39628-fc4c-4795-92d9-b09f14eadcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.044418 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc39628-fc4c-4795-92d9-b09f14eadcf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.045106 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc39628-fc4c-4795-92d9-b09f14eadcf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.047342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc39628-fc4c-4795-92d9-b09f14eadcf4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.048829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1288/2573] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:18.052803 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f6cdd-3130-41af-9243-ec48aec91c8c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.064090 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f6cdd-3130-41af-9243-ec48aec91c8c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.068897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7aead3-693f-41cb-8906-4040759f32c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.072720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7aead3-693f-41cb-8906-4040759f32c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.072720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7aead3-693f-41cb-8906-4040759f32c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.072989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1286/2574] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:17 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:18.073439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7aead3-693f-41cb-8906-4040759f32c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.073439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7aead3-693f-41cb-8906-4040759f32c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.073439 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7aead3-693f-41cb-8906-4040759f32c6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.073935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc39628-fc4c-4795-92d9-b09f14eadcf4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.087839 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1289/2575] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:18.089060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7aead3-693f-41cb-8906-4040759f32c6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.093652 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc39628-fc4c-4795-92d9-b09f14eadcf4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.100115 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7aead3-693f-41cb-8906-4040759f32c6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.104314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bb5cf-c77f-4e9f-b9b9-c513c95298d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.104993 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb5cf-c77f-4e9f-b9b9-c513c95298d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.105328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb5cf-c77f-4e9f-b9b9-c513c95298d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.105563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bb5cf-c77f-4e9f-b9b9-c513c95298d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.106093 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de47dfa-d670-4e93-af7a-25a007366b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.106290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bb5cf-c77f-4e9f-b9b9-c513c95298d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.107600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de47dfa-d670-4e93-af7a-25a007366b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.107885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de47dfa-d670-4e93-af7a-25a007366b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.108165 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de47dfa-d670-4e93-af7a-25a007366b39 admin admin] PATH_INFO: `/v3/projects/7e5a808ba59741f2820f988700f0470a/users/c1faf06ce90c4cf78c816e0a71a40b43/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.108533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bb5cf-c77f-4e9f-b9b9-c513c95298d5 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.127067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1287/2576] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:18.127385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de47dfa-d670-4e93-af7a-25a007366b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5a808ba59741f2820f988700f0470a, user_id=c1faf06ce90c4cf78c816e0a71a40b43, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.128984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de47dfa-d670-4e93-af7a-25a007366b39 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.130077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bb5cf-c77f-4e9f-b9b9-c513c95298d5 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.142242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bb5cf-c77f-4e9f-b9b9-c513c95298d5 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.148529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1290/2577] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:18 2022] PUT /identity/v3/projects/7e5a808ba59741f2820f988700f0470a/users/c1faf06ce90c4cf78c816e0a71a40b43/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:18.162958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f170f3-cd6a-4982-83ad-fd5935f25339 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.164703 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f170f3-cd6a-4982-83ad-fd5935f25339 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.165397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f170f3-cd6a-4982-83ad-fd5935f25339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.198569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:18.205621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-25f170f3-cd6a-4982-83ad-fd5935f25339 None None] MFA Rules not processed for user `c1faf06ce90c4cf78c816e0a71a40b43`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:18.206894 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-25f170f3-cd6a-4982-83ad-fd5935f25339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:18.231178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:18.236681 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e988add-fe6e-4c78-b388-1676f7af4f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.237364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e988add-fe6e-4c78-b388-1676f7af4f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.237641 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e988add-fe6e-4c78-b388-1676f7af4f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.237921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e988add-fe6e-4c78-b388-1676f7af4f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.238692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e988add-fe6e-4c78-b388-1676f7af4f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.240829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e988add-fe6e-4c78-b388-1676f7af4f31 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.263444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1288/2578] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:18.265151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e988add-fe6e-4c78-b388-1676f7af4f31 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.280104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e988add-fe6e-4c78-b388-1676f7af4f31 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.285107 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869dd48-22b2-4794-bd04-a9efa6b15104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.285936 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869dd48-22b2-4794-bd04-a9efa6b15104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.286284 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869dd48-22b2-4794-bd04-a9efa6b15104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.286592 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869dd48-22b2-4794-bd04-a9efa6b15104 admin admin] PATH_INFO: `/v3/projects/019ee7a13b8b4a3ba4d64ebed21ac58a/users/b8cc88ae4e9349e3aba51c977409e275/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.307122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869dd48-22b2-4794-bd04-a9efa6b15104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=019ee7a13b8b4a3ba4d64ebed21ac58a, user_id=b8cc88ae4e9349e3aba51c977409e275, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.310129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869dd48-22b2-4794-bd04-a9efa6b15104 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.318677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1291/2579] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:18.323622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f170f3-cd6a-4982-83ad-fd5935f25339 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.327442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1289/2580] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:18 2022] PUT /identity/v3/projects/019ee7a13b8b4a3ba4d64ebed21ac58a/users/b8cc88ae4e9349e3aba51c977409e275/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:18.336896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f170f3-cd6a-4982-83ad-fd5935f25339 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.339527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc96ab-09fa-4758-9c5e-ce492863f95f None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.339824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc96ab-09fa-4758-9c5e-ce492863f95f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.340108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc96ab-09fa-4758-9c5e-ce492863f95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.372188 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-f7cc96ab-09fa-4758-9c5e-ce492863f95f None None] MFA Rules not processed for user `b8cc88ae4e9349e3aba51c977409e275`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:18.373666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f7cc96ab-09fa-4758-9c5e-ce492863f95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:18.411882 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68abbe-9e48-453e-b59a-0316e580f5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.412780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68abbe-9e48-453e-b59a-0316e580f5a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.413356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68abbe-9e48-453e-b59a-0316e580f5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.413767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68abbe-9e48-453e-b59a-0316e580f5a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.414734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68abbe-9e48-453e-b59a-0316e580f5a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.417399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f68abbe-9e48-453e-b59a-0316e580f5a0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.452543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1290/2581] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:18.457113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc96ab-09fa-4758-9c5e-ce492863f95f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.457272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1292/2582] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:18 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:18.461241 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68abbe-9e48-453e-b59a-0316e580f5a0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.468861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc96ab-09fa-4758-9c5e-ce492863f95f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.530030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68abbe-9e48-453e-b59a-0316e580f5a0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.532671 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56716c-9619-4465-a4d9-6e61db889943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.533707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56716c-9619-4465-a4d9-6e61db889943 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.534108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56716c-9619-4465-a4d9-6e61db889943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.534503 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56716c-9619-4465-a4d9-6e61db889943 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.534877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba7538-39c8-42d0-911c-f2fcda4c9121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.535598 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba7538-39c8-42d0-911c-f2fcda4c9121 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.535705 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56716c-9619-4465-a4d9-6e61db889943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.535834 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba7538-39c8-42d0-911c-f2fcda4c9121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.536112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba7538-39c8-42d0-911c-f2fcda4c9121 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.536802 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba7538-39c8-42d0-911c-f2fcda4c9121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.538389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56716c-9619-4465-a4d9-6e61db889943 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.538762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba7538-39c8-42d0-911c-f2fcda4c9121 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.562026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1293/2583] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:18 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:18.563267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba7538-39c8-42d0-911c-f2fcda4c9121 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.574119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1291/2584] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:18 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:18.575905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba7538-39c8-42d0-911c-f2fcda4c9121 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.579383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56716c-9619-4465-a4d9-6e61db889943 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.621267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:18.630723 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:18.658931 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d9ddc-d1d0-487a-9126-6bb6c5a2d41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.659629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d9ddc-d1d0-487a-9126-6bb6c5a2d41b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.659916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d9ddc-d1d0-487a-9126-6bb6c5a2d41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.660203 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d9ddc-d1d0-487a-9126-6bb6c5a2d41b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.662722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d9ddc-d1d0-487a-9126-6bb6c5a2d41b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.664428 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d9ddc-d1d0-487a-9126-6bb6c5a2d41b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.681395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1294/2585] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:18.685263 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d9ddc-d1d0-487a-9126-6bb6c5a2d41b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.692346 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56716c-9619-4465-a4d9-6e61db889943 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.698917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b3514-e595-43ce-a8f4-919b292d73a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.699856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b3514-e595-43ce-a8f4-919b292d73a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.700660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b3514-e595-43ce-a8f4-919b292d73a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.700939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b3514-e595-43ce-a8f4-919b292d73a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.701636 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b3514-e595-43ce-a8f4-919b292d73a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.703631 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b3514-e595-43ce-a8f4-919b292d73a4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.708875 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d9ddc-d1d0-487a-9126-6bb6c5a2d41b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.721125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02a7fc-ca80-4ecf-a72b-9c60c04c76c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.722288 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02a7fc-ca80-4ecf-a72b-9c60c04c76c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.722605 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02a7fc-ca80-4ecf-a72b-9c60c04c76c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.722936 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02a7fc-ca80-4ecf-a72b-9c60c04c76c6 admin admin] PATH_INFO: `/v3/projects/a76c91f086784094b6297c01fa2c0ad9/users/158d5eba7c6f4a24a04c2e5abfcbc757/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.736966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1292/2586] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:18 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:18.741762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b3514-e595-43ce-a8f4-919b292d73a4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.743171 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02a7fc-ca80-4ecf-a72b-9c60c04c76c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a76c91f086784094b6297c01fa2c0ad9, user_id=158d5eba7c6f4a24a04c2e5abfcbc757, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.746133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02a7fc-ca80-4ecf-a72b-9c60c04c76c6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.754017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b3514-e595-43ce-a8f4-919b292d73a4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.766842 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1295/2587] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:18 2022] PUT /identity/v3/projects/a76c91f086784094b6297c01fa2c0ad9/users/158d5eba7c6f4a24a04c2e5abfcbc757/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:18.771566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02a7fc-ca80-4ecf-a72b-9c60c04c76c6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.842069 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc3d35-f588-4ad3-9925-026191c5dcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.842862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc3d35-f588-4ad3-9925-026191c5dcca admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.843147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc3d35-f588-4ad3-9925-026191c5dcca admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.843511 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc3d35-f588-4ad3-9925-026191c5dcca admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.844148 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc3d35-f588-4ad3-9925-026191c5dcca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.845916 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc3d35-f588-4ad3-9925-026191c5dcca admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.867607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1293/2588] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:18.868541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02a7fc-ca80-4ecf-a72b-9c60c04c76c6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.868818 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc3d35-f588-4ad3-9925-026191c5dcca admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.875238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942227a1-a117-4799-96b9-66cbacb3b568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.875921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942227a1-a117-4799-96b9-66cbacb3b568 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.876219 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942227a1-a117-4799-96b9-66cbacb3b568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.876493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942227a1-a117-4799-96b9-66cbacb3b568 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.877230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942227a1-a117-4799-96b9-66cbacb3b568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.879043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942227a1-a117-4799-96b9-66cbacb3b568 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.881172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc3d35-f588-4ad3-9925-026191c5dcca admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.890013 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237a33c-363e-4120-a30d-7082cf8a6726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.890789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237a33c-363e-4120-a30d-7082cf8a6726 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.891120 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237a33c-363e-4120-a30d-7082cf8a6726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.891432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237a33c-363e-4120-a30d-7082cf8a6726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.891654 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1296/2589] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:18.892172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237a33c-363e-4120-a30d-7082cf8a6726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.893879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237a33c-363e-4120-a30d-7082cf8a6726 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.895748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942227a1-a117-4799-96b9-66cbacb3b568 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.907500 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1294/2590] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:18.907724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942227a1-a117-4799-96b9-66cbacb3b568 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.911482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237a33c-363e-4120-a30d-7082cf8a6726 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:18.912531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925932c-4f43-42cc-8e1d-fbd43c182bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.913429 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925932c-4f43-42cc-8e1d-fbd43c182bed admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.913718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925932c-4f43-42cc-8e1d-fbd43c182bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.913998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925932c-4f43-42cc-8e1d-fbd43c182bed admin admin] PATH_INFO: `/v3/projects/a76c91f086784094b6297c01fa2c0ad9/users/158d5eba7c6f4a24a04c2e5abfcbc757/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.916266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925932c-4f43-42cc-8e1d-fbd43c182bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=a76c91f086784094b6297c01fa2c0ad9, user_id=158d5eba7c6f4a24a04c2e5abfcbc757, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.919089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5925932c-4f43-42cc-8e1d-fbd43c182bed admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.922967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237a33c-363e-4120-a30d-7082cf8a6726 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:18.928336 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d560e7a-e75b-495d-9aaf-96d347475893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:18.929338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d560e7a-e75b-495d-9aaf-96d347475893 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.929717 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d560e7a-e75b-495d-9aaf-96d347475893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.930190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d560e7a-e75b-495d-9aaf-96d347475893 admin admin] PATH_INFO: `/v3/projects/fbd56786315141fdb46592e68a884472/users/37d4584d31cb4ba484b11fbed5215198/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.930647 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1297/2591] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:18 2022] PUT /identity/v3/projects/a76c91f086784094b6297c01fa2c0ad9/users/158d5eba7c6f4a24a04c2e5abfcbc757/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:18.941527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ac5cb-62d9-462e-b19e-f633e1c3c6bf None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.942852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ac5cb-62d9-462e-b19e-f633e1c3c6bf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.943157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ac5cb-62d9-462e-b19e-f633e1c3c6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:18.953183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d560e7a-e75b-495d-9aaf-96d347475893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd56786315141fdb46592e68a884472, user_id=37d4584d31cb4ba484b11fbed5215198, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:18.956535 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d560e7a-e75b-495d-9aaf-96d347475893 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:18.973731 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-952ac5cb-62d9-462e-b19e-f633e1c3c6bf None None] MFA Rules not processed for user `158d5eba7c6f4a24a04c2e5abfcbc757`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:18.974685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1295/2592] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:18 2022] PUT /identity/v3/projects/fbd56786315141fdb46592e68a884472/users/37d4584d31cb4ba484b11fbed5215198/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:18.975447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-952ac5cb-62d9-462e-b19e-f633e1c3c6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:18.984886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf3ed7-5790-4b0f-a9a9-ffe1677be047 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:18.985169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf3ed7-5790-4b0f-a9a9-ffe1677be047 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:18.985445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf3ed7-5790-4b0f-a9a9-ffe1677be047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:19.012807 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e4bf3ed7-5790-4b0f-a9a9-ffe1677be047 None None] MFA Rules not processed for user `37d4584d31cb4ba484b11fbed5215198`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:19.014628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e4bf3ed7-5790-4b0f-a9a9-ffe1677be047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:19.050403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1298/2593] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:18 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:19.062938 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ac5cb-62d9-462e-b19e-f633e1c3c6bf None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:19.069851 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-952ac5cb-62d9-462e-b19e-f633e1c3c6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:19.078660 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:19.083922 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1296/2594] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:19.085603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:19.088918 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf3ed7-5790-4b0f-a9a9-ffe1677be047 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:19.132728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ac5cb-62d9-462e-b19e-f633e1c3c6bf None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:19.136699 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa3702-3f2d-476b-8a20-fdaa177d9a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:19.137296 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa3702-3f2d-476b-8a20-fdaa177d9a83 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:19.137603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa3702-3f2d-476b-8a20-fdaa177d9a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:19.138392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa3702-3f2d-476b-8a20-fdaa177d9a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:19.147891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa3702-3f2d-476b-8a20-fdaa177d9a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:19.149330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa3702-3f2d-476b-8a20-fdaa177d9a83 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:19.152936 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf3ed7-5790-4b0f-a9a9-ffe1677be047 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:19.158114 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeb020-f834-4fdc-a23f-f79359c49f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:19.158811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeb020-f834-4fdc-a23f-f79359c49f3b admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:19.159090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeb020-f834-4fdc-a23f-f79359c49f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:19.159361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeb020-f834-4fdc-a23f-f79359c49f3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:19.160034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1299/2595] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:19 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:34:19.160146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baeb020-f834-4fdc-a23f-f79359c49f3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:19.163969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baeb020-f834-4fdc-a23f-f79359c49f3b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:19.195592 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1297/2596] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:19 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:19.195917 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa3702-3f2d-476b-8a20-fdaa177d9a83 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:19.207083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa3702-3f2d-476b-8a20-fdaa177d9a83 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:19.236015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeb020-f834-4fdc-a23f-f79359c49f3b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:19.272504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcac26c-9916-4246-997d-9dd5df170460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:19.273385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcac26c-9916-4246-997d-9dd5df170460 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:19.273688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcac26c-9916-4246-997d-9dd5df170460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:19.274181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcac26c-9916-4246-997d-9dd5df170460 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:19.274898 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcac26c-9916-4246-997d-9dd5df170460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:19.276858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcac26c-9916-4246-997d-9dd5df170460 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:19.295444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeb020-f834-4fdc-a23f-f79359c49f3b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:19.299393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1300/2597] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:19 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:19.300129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c3ac-3c1f-4dcc-be3c-936776b6d67e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:19.300750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c3ac-3c1f-4dcc-be3c-936776b6d67e service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:19.301031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c3ac-3c1f-4dcc-be3c-936776b6d67e service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:19.301331 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032c3ac-3c1f-4dcc-be3c-936776b6d67e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:19.303689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcac26c-9916-4246-997d-9dd5df170460 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:19.321210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcac26c-9916-4246-997d-9dd5df170460 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:19.326608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dda2b0-ceb1-49a3-aa9a-7bd23fcad6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:19.327287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dda2b0-ceb1-49a3-aa9a-7bd23fcad6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:19.327575 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dda2b0-ceb1-49a3-aa9a-7bd23fcad6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:19.327854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dda2b0-ceb1-49a3-aa9a-7bd23fcad6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:19.328583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dda2b0-ceb1-49a3-aa9a-7bd23fcad6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:19.330203 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dda2b0-ceb1-49a3-aa9a-7bd23fcad6d0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:19.343777 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1301/2598] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:19.344824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dda2b0-ceb1-49a3-aa9a-7bd23fcad6d0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:19.355099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dda2b0-ceb1-49a3-aa9a-7bd23fcad6d0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:19.360424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51a3a-c741-4579-9e18-f0c0da94cf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:19.361304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51a3a-c741-4579-9e18-f0c0da94cf2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:19.361584 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51a3a-c741-4579-9e18-f0c0da94cf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:19.361856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e51a3a-c741-4579-9e18-f0c0da94cf2c admin admin] PATH_INFO: `/v3/projects/fcd91e6d3a3643d9aa0e6439112758d6/users/7d2802c8664a4db5a491953a4724a6b3/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:19.375221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032c3ac-3c1f-4dcc-be3c-936776b6d67e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:19.376638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032c3ac-3c1f-4dcc-be3c-936776b6d67e service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:19.380194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e51a3a-c741-4579-9e18-f0c0da94cf2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd91e6d3a3643d9aa0e6439112758d6, user_id=7d2802c8664a4db5a491953a4724a6b3, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:19.382941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e51a3a-c741-4579-9e18-f0c0da94cf2c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:19.390581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1298/2599] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:19 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:34:19.410321 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1302/2600] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:19 2022] PUT /identity/v3/projects/fcd91e6d3a3643d9aa0e6439112758d6/users/7d2802c8664a4db5a491953a4724a6b3/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:19.413987 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c3ac-3c1f-4dcc-be3c-936776b6d67e service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:19.430441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032c3ac-3c1f-4dcc-be3c-936776b6d67e service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:19.479586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:19.528396 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac28777-9031-40d6-98de-53abacb99c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:19.529087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac28777-9031-40d6-98de-53abacb99c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:19.529385 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac28777-9031-40d6-98de-53abacb99c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:19.529718 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac28777-9031-40d6-98de-53abacb99c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:19.530467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac28777-9031-40d6-98de-53abacb99c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:19.533192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac28777-9031-40d6-98de-53abacb99c91 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:19.553254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1299/2601] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:19.554823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51a3a-c741-4579-9e18-f0c0da94cf2c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:19.570030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e51a3a-c741-4579-9e18-f0c0da94cf2c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:19.576580 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8e982-a200-4a9e-9f46-70ae19bb1962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:19.577864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8e982-a200-4a9e-9f46-70ae19bb1962 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:19.577974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8e982-a200-4a9e-9f46-70ae19bb1962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:19.578225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8e982-a200-4a9e-9f46-70ae19bb1962 admin admin] PATH_INFO: `/v3/projects/fcd91e6d3a3643d9aa0e6439112758d6/users/7d2802c8664a4db5a491953a4724a6b3/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:19.581157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8e982-a200-4a9e-9f46-70ae19bb1962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd91e6d3a3643d9aa0e6439112758d6, user_id=7d2802c8664a4db5a491953a4724a6b3, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:19.584167 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d8e982-a200-4a9e-9f46-70ae19bb1962 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:19.597185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1303/2602] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:19 2022] PUT /identity/v3/projects/fcd91e6d3a3643d9aa0e6439112758d6/users/7d2802c8664a4db5a491953a4724a6b3/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:19.609350 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fa3ba-fd2f-46ac-be47-7175f23c6102 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:19.609647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fa3ba-fd2f-46ac-be47-7175f23c6102 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:19.609932 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183fa3ba-fd2f-46ac-be47-7175f23c6102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:19.638755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-183fa3ba-fd2f-46ac-be47-7175f23c6102 None None] MFA Rules not processed for user `7d2802c8664a4db5a491953a4724a6b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:19.639992 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-183fa3ba-fd2f-46ac-be47-7175f23c6102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:19.716316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1300/2603] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:19.728713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8e982-a200-4a9e-9f46-70ae19bb1962 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:19.732274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c5d8e982-a200-4a9e-9f46-70ae19bb1962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:19.799230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8e982-a200-4a9e-9f46-70ae19bb1962 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:19.802888 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e13bed-f898-433f-a0c7-44384e4e9a6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:19.803512 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e13bed-f898-433f-a0c7-44384e4e9a6e service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:19.803793 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e13bed-f898-433f-a0c7-44384e4e9a6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:19.804066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e13bed-f898-433f-a0c7-44384e4e9a6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:19.813508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e13bed-f898-433f-a0c7-44384e4e9a6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:19.814998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e13bed-f898-433f-a0c7-44384e4e9a6e service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:19.820134 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:19.825868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1304/2604] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:20.378439 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fa3ba-fd2f-46ac-be47-7175f23c6102 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:20.388609 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183fa3ba-fd2f-46ac-be47-7175f23c6102 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:20.392324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413ca16-3caa-44e9-9deb-fcbe7b877935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:20.392965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413ca16-3caa-44e9-9deb-fcbe7b877935 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:20.393244 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413ca16-3caa-44e9-9deb-fcbe7b877935 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:20.393563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413ca16-3caa-44e9-9deb-fcbe7b877935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:20.459183 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413ca16-3caa-44e9-9deb-fcbe7b877935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:20.460682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413ca16-3caa-44e9-9deb-fcbe7b877935 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:20.470835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1301/2605] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:27.925307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e13bed-f898-433f-a0c7-44384e4e9a6e service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:27.934342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e13bed-f898-433f-a0c7-44384e4e9a6e service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:27.938478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0f102-8fce-4f28-936a-b5eb49151823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:27.939153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0f102-8fce-4f28-936a-b5eb49151823 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:27.939433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0f102-8fce-4f28-936a-b5eb49151823 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:27.939705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0f102-8fce-4f28-936a-b5eb49151823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:28.001805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0f102-8fce-4f28-936a-b5eb49151823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:28.004384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0f102-8fce-4f28-936a-b5eb49151823 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:28.018683 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1305/2606] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:30.383320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413ca16-3caa-44e9-9deb-fcbe7b877935 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:30.442048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:30.446233 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413ca16-3caa-44e9-9deb-fcbe7b877935 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:30.451731 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:30.452596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:30.452880 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:30.453153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] PATH_INFO: `/v3/users/b8cc88ae4e9349e3aba51c977409e275` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:30.454342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8cc88ae4e9349e3aba51c977409e275)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:30.456314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:30.513515 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8cc88ae4e9349e3aba51c977409e275'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:30.532456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8cc88ae4e9349e3aba51c977409e275'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:30.533187 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8cc88ae4e9349e3aba51c977409e275'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:30.546151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8cc88ae4e9349e3aba51c977409e275'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:30.546929 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8cc88ae4e9349e3aba51c977409e275'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:30.563288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1302/2607] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:30 2022] DELETE /identity/v3/users/b8cc88ae4e9349e3aba51c977409e275 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:30.567820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573bdc5-36bf-401c-a9ee-c4ab04ae23c3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:30.568302 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573bdc5-36bf-401c-a9ee-c4ab04ae23c3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:30.568550 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573bdc5-36bf-401c-a9ee-c4ab04ae23c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:30.586349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-6573bdc5-36bf-401c-a9ee-c4ab04ae23c3 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:30.587680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6573bdc5-36bf-401c-a9ee-c4ab04ae23c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:30.654621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1306/2608] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:30.661819 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:30.663276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:30.730602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c2eda-39e6-4e5f-9dfb-77d006940e65 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:30.734308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc40a6-bf5a-4025-a9c7-8b0f797dda5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:30.734983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc40a6-bf5a-4025-a9c7-8b0f797dda5e service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:30.735267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc40a6-bf5a-4025-a9c7-8b0f797dda5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:30.735540 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc40a6-bf5a-4025-a9c7-8b0f797dda5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:30.744584 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc40a6-bf5a-4025-a9c7-8b0f797dda5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:30.746030 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc40a6-bf5a-4025-a9c7-8b0f797dda5e service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:30.755435 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1303/2609] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:31.029937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573bdc5-36bf-401c-a9ee-c4ab04ae23c3 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:31.031559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6573bdc5-36bf-401c-a9ee-c4ab04ae23c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:31.043963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573bdc5-36bf-401c-a9ee-c4ab04ae23c3 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:31.049448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:31.050315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:31.050621 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:31.050915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] PATH_INFO: `/v3/projects/019ee7a13b8b4a3ba4d64ebed21ac58a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:31.052608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=019ee7a13b8b4a3ba4d64ebed21ac58a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:31.055029 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:31.106521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '019ee7a13b8b4a3ba4d64ebed21ac58a', 'request_id': 'req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.117350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '019ee7a13b8b4a3ba4d64ebed21ac58a', 'request_id': 'req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.127133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '019ee7a13b8b4a3ba4d64ebed21ac58a', 'request_id': 'req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.142326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] The token cache is being invalidate because project 019ee7a13b8b4a3ba4d64ebed21ac58a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:31.142662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.143344 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.144901 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1307/2610] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:31 2022] DELETE /identity/v3/projects/019ee7a13b8b4a3ba4d64ebed21ac58a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:31.152104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc40a6-bf5a-4025-a9c7-8b0f797dda5e service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:31.166919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc40a6-bf5a-4025-a9c7-8b0f797dda5e service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:31.244501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a734d-80b4-4dd8-9c9e-4e5fb5dec2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:31.245392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a734d-80b4-4dd8-9c9e-4e5fb5dec2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:31.245673 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a734d-80b4-4dd8-9c9e-4e5fb5dec2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:31.245948 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a734d-80b4-4dd8-9c9e-4e5fb5dec2a6 admin admin] PATH_INFO: `/v3/users/37d4584d31cb4ba484b11fbed5215198` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:31.247280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a734d-80b4-4dd8-9c9e-4e5fb5dec2a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37d4584d31cb4ba484b11fbed5215198)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:31.249246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a734d-80b4-4dd8-9c9e-4e5fb5dec2a6 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:31.264711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:31.266131 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:31.279420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:31.296607 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d22c3-dd81-4302-b87d-bcb8bc7cd368 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:31.301956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:31.303450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:31.303450 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:31.303767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] PATH_INFO: `/v3/users/c1faf06ce90c4cf78c816e0a71a40b43` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:31.304806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1faf06ce90c4cf78c816e0a71a40b43)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:31.306808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:31.319377 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-132a734d-80b4-4dd8-9c9e-4e5fb5dec2a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37d4584d31cb4ba484b11fbed5215198'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.327235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:31.334649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-132a734d-80b4-4dd8-9c9e-4e5fb5dec2a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37d4584d31cb4ba484b11fbed5215198'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.335401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-132a734d-80b4-4dd8-9c9e-4e5fb5dec2a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37d4584d31cb4ba484b11fbed5215198'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.343986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-132a734d-80b4-4dd8-9c9e-4e5fb5dec2a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37d4584d31cb4ba484b11fbed5215198'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.344953 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-132a734d-80b4-4dd8-9c9e-4e5fb5dec2a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37d4584d31cb4ba484b11fbed5215198'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.365513 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1304/2611] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:31 2022] DELETE /identity/v3/users/37d4584d31cb4ba484b11fbed5215198 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:31.373010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1faf06ce90c4cf78c816e0a71a40b43'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.391999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1faf06ce90c4cf78c816e0a71a40b43'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.392754 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1faf06ce90c4cf78c816e0a71a40b43'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.402524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1faf06ce90c4cf78c816e0a71a40b43'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.404059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1faf06ce90c4cf78c816e0a71a40b43'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.419595 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1308/2612] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:31 2022] DELETE /identity/v3/users/c1faf06ce90c4cf78c816e0a71a40b43 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:31.427909 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c9b55-7c3b-411b-97bc-88f4604989ae None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:31.428615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c9b55-7c3b-411b-97bc-88f4604989ae None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:31.428615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1c9b55-7c3b-411b-97bc-88f4604989ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:31.447380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-fb1c9b55-7c3b-411b-97bc-88f4604989ae None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:31.448622 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-fb1c9b55-7c3b-411b-97bc-88f4604989ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:31.525342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1305/2613] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:31.536367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:31.538365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:31.599500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f0f04-4511-4a9d-aa43-7198adb1d101 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:31.603469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6d5de-aff9-40e8-b5fb-a37174b6a474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:31.604202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6d5de-aff9-40e8-b5fb-a37174b6a474 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:31.604556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6d5de-aff9-40e8-b5fb-a37174b6a474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:31.604903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a6d5de-aff9-40e8-b5fb-a37174b6a474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:31.615193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6d5de-aff9-40e8-b5fb-a37174b6a474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:31.616651 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a6d5de-aff9-40e8-b5fb-a37174b6a474 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:31.628350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1309/2614] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:31.670444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c9b55-7c3b-411b-97bc-88f4604989ae None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:31.677828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-fb1c9b55-7c3b-411b-97bc-88f4604989ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:31.692330 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1c9b55-7c3b-411b-97bc-88f4604989ae None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:31.709723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:31.710957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:31.711563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:31.711917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] PATH_INFO: `/v3/projects/fbd56786315141fdb46592e68a884472` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:31.713624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbd56786315141fdb46592e68a884472)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:31.724852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:31.812366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbd56786315141fdb46592e68a884472', 'request_id': 'req-da5ca58c-9c4c-41ab-9521-217b3638dec2'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.820623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbd56786315141fdb46592e68a884472', 'request_id': 'req-da5ca58c-9c4c-41ab-9521-217b3638dec2'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.833212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbd56786315141fdb46592e68a884472', 'request_id': 'req-da5ca58c-9c4c-41ab-9521-217b3638dec2'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.850457 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] The token cache is being invalidate because project fbd56786315141fdb46592e68a884472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:31.851027 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.851908 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:31.853822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1306/2615] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:31 2022] DELETE /identity/v3/projects/fbd56786315141fdb46592e68a884472 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:31.858127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6d5de-aff9-40e8-b5fb-a37174b6a474 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:31.869919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a6d5de-aff9-40e8-b5fb-a37174b6a474 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:31.937635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:31.938591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:31.938877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:31.939337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] PATH_INFO: `/v3/users/7d2802c8664a4db5a491953a4724a6b3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:31.940469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d2802c8664a4db5a491953a4724a6b3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:31.942613 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:31.988582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:31.990304 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:32.000831 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:32.009903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ca58c-9c4c-41ab-9521-217b3638dec2 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:32.016645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:32.018056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:32.018907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:32.019366 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] PATH_INFO: `/v3/projects/7e5a808ba59741f2820f988700f0470a` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:32.021955 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5a808ba59741f2820f988700f0470a)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:32.023521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d2802c8664a4db5a491953a4724a6b3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.025380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:32.042614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d2802c8664a4db5a491953a4724a6b3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.044392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d2802c8664a4db5a491953a4724a6b3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.051043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:32.054644 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d2802c8664a4db5a491953a4724a6b3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.055588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d2802c8664a4db5a491953a4724a6b3'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.071692 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1310/2616] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:31 2022] DELETE /identity/v3/users/7d2802c8664a4db5a491953a4724a6b3 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:32.096849 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5a808ba59741f2820f988700f0470a', 'request_id': 'req-7456ec98-5ad8-4f1d-8223-20629f038ec1'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.105699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5a808ba59741f2820f988700f0470a', 'request_id': 'req-7456ec98-5ad8-4f1d-8223-20629f038ec1'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.119095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5a808ba59741f2820f988700f0470a', 'request_id': 'req-7456ec98-5ad8-4f1d-8223-20629f038ec1'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.134282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] The token cache is being invalidate because project 7e5a808ba59741f2820f988700f0470a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:32.134609 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.135688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.140582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1307/2617] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:31 2022] DELETE /identity/v3/projects/7e5a808ba59741f2820f988700f0470a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:32.141893 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:32.143435 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:32.239319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49112056-eddb-4e3c-a94f-24e3d286f3e9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:32.331767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12ff41-e2f7-4abe-9c47-d6619b253878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:32.332737 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ff41-e2f7-4abe-9c47-d6619b253878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:32.333178 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ff41-e2f7-4abe-9c47-d6619b253878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:32.333479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ff41-e2f7-4abe-9c47-d6619b253878 admin admin] PATH_INFO: `/v3/users/158d5eba7c6f4a24a04c2e5abfcbc757` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:32.334837 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12ff41-e2f7-4abe-9c47-d6619b253878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=158d5eba7c6f4a24a04c2e5abfcbc757)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:32.336953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e12ff41-e2f7-4abe-9c47-d6619b253878 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:32.407075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9e12ff41-e2f7-4abe-9c47-d6619b253878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '158d5eba7c6f4a24a04c2e5abfcbc757'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.422434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9e12ff41-e2f7-4abe-9c47-d6619b253878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '158d5eba7c6f4a24a04c2e5abfcbc757'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.423306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9e12ff41-e2f7-4abe-9c47-d6619b253878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '158d5eba7c6f4a24a04c2e5abfcbc757'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.424868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:32.435944 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9e12ff41-e2f7-4abe-9c47-d6619b253878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '158d5eba7c6f4a24a04c2e5abfcbc757'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.437890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-9e12ff41-e2f7-4abe-9c47-d6619b253878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '158d5eba7c6f4a24a04c2e5abfcbc757'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.454776 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:32.457924 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:32.472254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456ec98-5ad8-4f1d-8223-20629f038ec1 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:32.477324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1311/2618] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:32 2022] DELETE /identity/v3/users/158d5eba7c6f4a24a04c2e5abfcbc757 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:32.561847 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:32.562865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:32.563141 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:32.563421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] PATH_INFO: `/v3/projects/fcd91e6d3a3643d9aa0e6439112758d6` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:32.564655 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd91e6d3a3643d9aa0e6439112758d6)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:32.566621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:32.628428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd91e6d3a3643d9aa0e6439112758d6', 'request_id': 'req-3b1d538c-1184-4844-81a8-cc72f76c0eb4'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.639499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd91e6d3a3643d9aa0e6439112758d6', 'request_id': 'req-3b1d538c-1184-4844-81a8-cc72f76c0eb4'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.658427 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd91e6d3a3643d9aa0e6439112758d6', 'request_id': 'req-3b1d538c-1184-4844-81a8-cc72f76c0eb4'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.681940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] The token cache is being invalidate because project fcd91e6d3a3643d9aa0e6439112758d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:32.682222 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.682951 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3b1d538c-1184-4844-81a8-cc72f76c0eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.685485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1308/2619] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:32 2022] DELETE /identity/v3/projects/fcd91e6d3a3643d9aa0e6439112758d6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:32.691673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed94570-3201-4569-83b7-279b815eb900 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:32.692840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed94570-3201-4569-83b7-279b815eb900 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:32.692840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed94570-3201-4569-83b7-279b815eb900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:32.712638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-bed94570-3201-4569-83b7-279b815eb900 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:32.713862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bed94570-3201-4569-83b7-279b815eb900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:32.805969 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1312/2620] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:32.816810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e956fb9-4029-45c7-815c-3caa2ede683c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:32.817139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e956fb9-4029-45c7-815c-3caa2ede683c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:32.817410 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e956fb9-4029-45c7-815c-3caa2ede683c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:32.831358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed94570-3201-4569-83b7-279b815eb900 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:32.840525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-bed94570-3201-4569-83b7-279b815eb900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:32.854281 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed94570-3201-4569-83b7-279b815eb900 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:32.861825 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:32.863047 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:32.863521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:32.864198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] PATH_INFO: `/v3/projects/a76c91f086784094b6297c01fa2c0ad9` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:32.866660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a76c91f086784094b6297c01fa2c0ad9)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:32.870132 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:32.875670 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-6e956fb9-4029-45c7-815c-3caa2ede683c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:32.877856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6e956fb9-4029-45c7-815c-3caa2ede683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:32.892981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1309/2621] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:32.894938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e956fb9-4029-45c7-815c-3caa2ede683c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:32.906642 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e956fb9-4029-45c7-815c-3caa2ede683c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:32.911889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8797d7e-863f-4131-bc1d-a519b398f060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:32.912649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8797d7e-863f-4131-bc1d-a519b398f060 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:32.912998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8797d7e-863f-4131-bc1d-a519b398f060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:32.913368 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8797d7e-863f-4131-bc1d-a519b398f060 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:32.914176 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8797d7e-863f-4131-bc1d-a519b398f060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:32.914575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8797d7e-863f-4131-bc1d-a519b398f060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:32.916686 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8797d7e-863f-4131-bc1d-a519b398f060 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:32.932372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1310/2622] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:34:32.937344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8797d7e-863f-4131-bc1d-a519b398f060 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:32.949373 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8797d7e-863f-4131-bc1d-a519b398f060 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:32.951524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a76c91f086784094b6297c01fa2c0ad9', 'request_id': 'req-7f13272c-4e5b-45d8-92ee-13cb350b076d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.959216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a76c91f086784094b6297c01fa2c0ad9', 'request_id': 'req-7f13272c-4e5b-45d8-92ee-13cb350b076d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.964392 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:32.973961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a76c91f086784094b6297c01fa2c0ad9', 'request_id': 'req-7f13272c-4e5b-45d8-92ee-13cb350b076d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.988628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] The token cache is being invalidate because project a76c91f086784094b6297c01fa2c0ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:32.989024 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.989705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7f13272c-4e5b-45d8-92ee-13cb350b076d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:32.991676 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1313/2623] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:32 2022] DELETE /identity/v3/projects/a76c91f086784094b6297c01fa2c0ad9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:32.997896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de7cf4-c77d-474e-a4ce-cd2d9ea71f09 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:32.998111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de7cf4-c77d-474e-a4ce-cd2d9ea71f09 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:32.998355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de7cf4-c77d-474e-a4ce-cd2d9ea71f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.018466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac956c-3eff-4a11-ae94-c19a1e49b254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.019276 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac956c-3eff-4a11-ae94-c19a1e49b254 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.019623 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac956c-3eff-4a11-ae94-c19a1e49b254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.019958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac956c-3eff-4a11-ae94-c19a1e49b254 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.020771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac956c-3eff-4a11-ae94-c19a1e49b254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.023397 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac956c-3eff-4a11-ae94-c19a1e49b254 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.024114 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-97de7cf4-c77d-474e-a4ce-cd2d9ea71f09 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:33.025846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-97de7cf4-c77d-474e-a4ce-cd2d9ea71f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:33.043810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1314/2624] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:33.047587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989758f-3068-47af-8d4e-8bb8e5898a83 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.048035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989758f-3068-47af-8d4e-8bb8e5898a83 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.048351 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9989758f-3068-47af-8d4e-8bb8e5898a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.060937 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1311/2625] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:32 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:33.065448 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac956c-3eff-4a11-ae94-c19a1e49b254 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.067482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-cdac956c-3eff-4a11-ae94-c19a1e49b254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:33.070954 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-9989758f-3068-47af-8d4e-8bb8e5898a83 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:33.072255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-9989758f-3068-47af-8d4e-8bb8e5898a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:33.127537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:33.137216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac956c-3eff-4a11-ae94-c19a1e49b254 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.140007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1315/2626] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:33.142233 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff43c0-39c4-48c8-8ffb-1fd23cf01905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.142941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff43c0-39c4-48c8-8ffb-1fd23cf01905 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.143309 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff43c0-39c4-48c8-8ffb-1fd23cf01905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.143589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff43c0-39c4-48c8-8ffb-1fd23cf01905 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.144277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff43c0-39c4-48c8-8ffb-1fd23cf01905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.144501 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989758f-3068-47af-8d4e-8bb8e5898a83 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.146374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff43c0-39c4-48c8-8ffb-1fd23cf01905 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.154309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9989758f-3068-47af-8d4e-8bb8e5898a83 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.159913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701ccdc-56b4-43e2-b297-6cfd91a94df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.160763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701ccdc-56b4-43e2-b297-6cfd91a94df6 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.161045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701ccdc-56b4-43e2-b297-6cfd91a94df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.161349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701ccdc-56b4-43e2-b297-6cfd91a94df6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.162398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701ccdc-56b4-43e2-b297-6cfd91a94df6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:33.162853 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701ccdc-56b4-43e2-b297-6cfd91a94df6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.164482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701ccdc-56b4-43e2-b297-6cfd91a94df6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.171161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1312/2627] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:33 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:33.175973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff43c0-39c4-48c8-8ffb-1fd23cf01905 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.180823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1316/2628] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:34:33.185713 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701ccdc-56b4-43e2-b297-6cfd91a94df6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.187485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff43c0-39c4-48c8-8ffb-1fd23cf01905 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.192190 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872f3f2-384f-4bc1-ad9a-16c11e0e8dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.192850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872f3f2-384f-4bc1-ad9a-16c11e0e8dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.193318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872f3f2-384f-4bc1-ad9a-16c11e0e8dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.193508 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872f3f2-384f-4bc1-ad9a-16c11e0e8dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.194494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872f3f2-384f-4bc1-ad9a-16c11e0e8dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.195995 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872f3f2-384f-4bc1-ad9a-16c11e0e8dfe admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.196249 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701ccdc-56b4-43e2-b297-6cfd91a94df6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.201283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df658859-0d41-48af-ba60-3e6607a4e9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.201983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658859-0d41-48af-ba60-3e6607a4e9c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.202269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658859-0d41-48af-ba60-3e6607a4e9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.202579 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df658859-0d41-48af-ba60-3e6607a4e9c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.203282 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df658859-0d41-48af-ba60-3e6607a4e9c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.205216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df658859-0d41-48af-ba60-3e6607a4e9c5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.207320 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1313/2629] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:33.211012 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872f3f2-384f-4bc1-ad9a-16c11e0e8dfe admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.221598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872f3f2-384f-4bc1-ad9a-16c11e0e8dfe admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.227305 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110db40-ba82-4564-a88f-934f043bb5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.228064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110db40-ba82-4564-a88f-934f043bb5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.228692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110db40-ba82-4564-a88f-934f043bb5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.228692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110db40-ba82-4564-a88f-934f043bb5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.229568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110db40-ba82-4564-a88f-934f043bb5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.231306 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110db40-ba82-4564-a88f-934f043bb5d1 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.246804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1317/2630] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:33 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:33.247016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1314/2631] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:33.247118 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df658859-0d41-48af-ba60-3e6607a4e9c5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.248466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110db40-ba82-4564-a88f-934f043bb5d1 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.262685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110db40-ba82-4564-a88f-934f043bb5d1 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.317748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df658859-0d41-48af-ba60-3e6607a4e9c5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.322879 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc72b6-8006-42f9-bc19-95013ffd11ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.323039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31504056-4f4c-4d54-bb5e-7a395a6a65b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.323788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc72b6-8006-42f9-bc19-95013ffd11ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.323943 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31504056-4f4c-4d54-bb5e-7a395a6a65b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.324061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc72b6-8006-42f9-bc19-95013ffd11ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.324276 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31504056-4f4c-4d54-bb5e-7a395a6a65b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.324391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc72b6-8006-42f9-bc19-95013ffd11ef admin admin] PATH_INFO: `/v3/projects/04f6d0b781f549a8a3fab77c43d0e274/users/eebc0d4cf563484a8cec9b0ddad10937/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.324548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31504056-4f4c-4d54-bb5e-7a395a6a65b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.325244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31504056-4f4c-4d54-bb5e-7a395a6a65b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.327366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31504056-4f4c-4d54-bb5e-7a395a6a65b3 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.337159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:33.343051 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc72b6-8006-42f9-bc19-95013ffd11ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f6d0b781f549a8a3fab77c43d0e274, user_id=eebc0d4cf563484a8cec9b0ddad10937, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.346732 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc72b6-8006-42f9-bc19-95013ffd11ef admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.352806 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1318/2632] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:33 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:33.353854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31504056-4f4c-4d54-bb5e-7a395a6a65b3 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.364913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31504056-4f4c-4d54-bb5e-7a395a6a65b3 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.371444 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1315/2633] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:33 2022] PUT /identity/v3/projects/04f6d0b781f549a8a3fab77c43d0e274/users/eebc0d4cf563484a8cec9b0ddad10937/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:33.383772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de5d2c-6667-4f6c-9916-b40df3957c19 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.384258 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de5d2c-6667-4f6c-9916-b40df3957c19 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.384543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de5d2c-6667-4f6c-9916-b40df3957c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.425574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9a451-c349-4e78-9f7c-a2c8151927c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.426161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-75de5d2c-6667-4f6c-9916-b40df3957c19 None None] MFA Rules not processed for user `eebc0d4cf563484a8cec9b0ddad10937`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:33.426625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9a451-c349-4e78-9f7c-a2c8151927c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.426971 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9a451-c349-4e78-9f7c-a2c8151927c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.427353 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9a451-c349-4e78-9f7c-a2c8151927c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.427483 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-75de5d2c-6667-4f6c-9916-b40df3957c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:33.428273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9a451-c349-4e78-9f7c-a2c8151927c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.430039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9a451-c349-4e78-9f7c-a2c8151927c1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.440927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1319/2634] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:33.445365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9a451-c349-4e78-9f7c-a2c8151927c1 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.457752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9a451-c349-4e78-9f7c-a2c8151927c1 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.465188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8e95-5763-4f85-95c1-d7e54b28c27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.466682 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8e95-5763-4f85-95c1-d7e54b28c27f admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.467072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8e95-5763-4f85-95c1-d7e54b28c27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.467407 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a8e95-5763-4f85-95c1-d7e54b28c27f admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.469201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a8e95-5763-4f85-95c1-d7e54b28c27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.469995 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a8e95-5763-4f85-95c1-d7e54b28c27f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.480981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1320/2635] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:33.485564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8e95-5763-4f85-95c1-d7e54b28c27f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.497777 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a8e95-5763-4f85-95c1-d7e54b28c27f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.502942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6493b9-1970-4257-8ab4-c5a3b6eef6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.503914 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6493b9-1970-4257-8ab4-c5a3b6eef6e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.504187 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6493b9-1970-4257-8ab4-c5a3b6eef6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.504456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6493b9-1970-4257-8ab4-c5a3b6eef6e1 admin admin] PATH_INFO: `/v3/projects/b68656329cd44267a500f6b0eb75b182/users/bf06847e997d4bc8bda98d9aece6b487/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.515057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1316/2636] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:33.519765 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de5d2c-6667-4f6c-9916-b40df3957c19 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.526502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6493b9-1970-4257-8ab4-c5a3b6eef6e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68656329cd44267a500f6b0eb75b182, user_id=bf06847e997d4bc8bda98d9aece6b487, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.530008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6493b9-1970-4257-8ab4-c5a3b6eef6e1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.531402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de5d2c-6667-4f6c-9916-b40df3957c19 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.536157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0c58a-9657-40df-9d5b-4e44899db7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.536817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c58a-9657-40df-9d5b-4e44899db7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.537099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c58a-9657-40df-9d5b-4e44899db7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.537374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0c58a-9657-40df-9d5b-4e44899db7d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.538067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0c58a-9657-40df-9d5b-4e44899db7d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.540004 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0c58a-9657-40df-9d5b-4e44899db7d0 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.543900 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1321/2637] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:33 2022] PUT /identity/v3/projects/b68656329cd44267a500f6b0eb75b182/users/bf06847e997d4bc8bda98d9aece6b487/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:33.555044 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa636e27-d7b8-480d-a4bc-96825f34878b None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.555337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa636e27-d7b8-480d-a4bc-96825f34878b None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.555623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa636e27-d7b8-480d-a4bc-96825f34878b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.567364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1317/2638] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:33 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:33.572685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0c58a-9657-40df-9d5b-4e44899db7d0 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.585258 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-aa636e27-d7b8-480d-a4bc-96825f34878b None None] MFA Rules not processed for user `bf06847e997d4bc8bda98d9aece6b487`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:33.586796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-aa636e27-d7b8-480d-a4bc-96825f34878b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:33.591581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:33.659578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0c58a-9657-40df-9d5b-4e44899db7d0 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.665935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858d97d-fc22-48fb-b8b4-a64b7999ba56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.666846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858d97d-fc22-48fb-b8b4-a64b7999ba56 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.667043 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858d97d-fc22-48fb-b8b4-a64b7999ba56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.667382 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858d97d-fc22-48fb-b8b4-a64b7999ba56 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.668137 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858d97d-fc22-48fb-b8b4-a64b7999ba56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.670688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858d97d-fc22-48fb-b8b4-a64b7999ba56 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.697018 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1322/2639] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:33.698579 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1318/2640] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:33 2022] POST /identity/v3/users => generated 298 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:33.703193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa636e27-d7b8-480d-a4bc-96825f34878b None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.703434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858d97d-fc22-48fb-b8b4-a64b7999ba56 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.724306 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa636e27-d7b8-480d-a4bc-96825f34878b None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.734612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771593f-50a8-4369-a443-5b68d701598a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.735390 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771593f-50a8-4369-a443-5b68d701598a admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.735897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771593f-50a8-4369-a443-5b68d701598a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.736089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771593f-50a8-4369-a443-5b68d701598a admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.736501 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858d97d-fc22-48fb-b8b4-a64b7999ba56 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.736910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771593f-50a8-4369-a443-5b68d701598a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.742300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771593f-50a8-4369-a443-5b68d701598a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.742493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ba606-144b-4ebf-8733-fd185924ac06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.742493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ba606-144b-4ebf-8733-fd185924ac06 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.742493 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ba606-144b-4ebf-8733-fd185924ac06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.742975 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753ba606-144b-4ebf-8733-fd185924ac06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.743910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ba606-144b-4ebf-8733-fd185924ac06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.745537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753ba606-144b-4ebf-8733-fd185924ac06 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.762579 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1319/2641] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:33.765658 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ba606-144b-4ebf-8733-fd185924ac06 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.777694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753ba606-144b-4ebf-8733-fd185924ac06 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.786267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd66a0c-5057-4cb5-956f-737fa0a8cf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.786601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1323/2642] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:33 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:33.787834 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd66a0c-5057-4cb5-956f-737fa0a8cf2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.788292 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd66a0c-5057-4cb5-956f-737fa0a8cf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.788783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd66a0c-5057-4cb5-956f-737fa0a8cf2f admin admin] PATH_INFO: `/v3/projects/05f33a73e6b54cda8747d522a061577d/users/59692774954b4074a62a7cd4e666451a/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.790499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771593f-50a8-4369-a443-5b68d701598a admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.814453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd66a0c-5057-4cb5-956f-737fa0a8cf2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f33a73e6b54cda8747d522a061577d, user_id=59692774954b4074a62a7cd4e666451a, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.818633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd66a0c-5057-4cb5-956f-737fa0a8cf2f admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.846987 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1320/2643] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:33 2022] PUT /identity/v3/projects/05f33a73e6b54cda8747d522a061577d/users/59692774954b4074a62a7cd4e666451a/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:33.850363 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd66a0c-5057-4cb5-956f-737fa0a8cf2f admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.867203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:33.892962 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771593f-50a8-4369-a443-5b68d701598a admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.931547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd66a0c-5057-4cb5-956f-737fa0a8cf2f admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.939553 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab817e88-31de-4536-b2b9-51b5f5418185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.940337 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817e88-31de-4536-b2b9-51b5f5418185 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.940691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817e88-31de-4536-b2b9-51b5f5418185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.941039 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab817e88-31de-4536-b2b9-51b5f5418185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.941872 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab817e88-31de-4536-b2b9-51b5f5418185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.943685 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab817e88-31de-4536-b2b9-51b5f5418185 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.954933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1321/2644] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:33.959907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab817e88-31de-4536-b2b9-51b5f5418185 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:33.963159 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3825ed-2dde-47e1-a0af-bfde6c9df8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.963922 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3825ed-2dde-47e1-a0af-bfde6c9df8ee admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.964273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3825ed-2dde-47e1-a0af-bfde6c9df8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.964620 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3825ed-2dde-47e1-a0af-bfde6c9df8ee admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.965385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3825ed-2dde-47e1-a0af-bfde6c9df8ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.967508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3825ed-2dde-47e1-a0af-bfde6c9df8ee admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.970062 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab817e88-31de-4536-b2b9-51b5f5418185 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:33.975037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be3387-93d2-4ff2-81e4-c230e51b1871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:33.975920 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be3387-93d2-4ff2-81e4-c230e51b1871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:33.976636 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be3387-93d2-4ff2-81e4-c230e51b1871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:33.976636 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be3387-93d2-4ff2-81e4-c230e51b1871 admin admin] PATH_INFO: `/v3/projects/05f33a73e6b54cda8747d522a061577d/users/59692774954b4074a62a7cd4e666451a/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:33.978778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be3387-93d2-4ff2-81e4-c230e51b1871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05f33a73e6b54cda8747d522a061577d, user_id=59692774954b4074a62a7cd4e666451a, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:33.981524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be3387-93d2-4ff2-81e4-c230e51b1871 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:33.991750 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1324/2645] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:33 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:33.992619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1322/2646] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:33 2022] PUT /identity/v3/projects/05f33a73e6b54cda8747d522a061577d/users/59692774954b4074a62a7cd4e666451a/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:33.992775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3825ed-2dde-47e1-a0af-bfde6c9df8ee admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:34.003261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3825ed-2dde-47e1-a0af-bfde6c9df8ee admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:34.004189 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0467f-4412-4b4a-96cb-e8b99f84eccb None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:34.004480 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0467f-4412-4b4a-96cb-e8b99f84eccb None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:34.004766 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0467f-4412-4b4a-96cb-e8b99f84eccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:34.024393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:34.033408 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d7f0467f-4412-4b4a-96cb-e8b99f84eccb None None] MFA Rules not processed for user `59692774954b4074a62a7cd4e666451a`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:34.034665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d7f0467f-4412-4b4a-96cb-e8b99f84eccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:34.070461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca203a-bdfd-4c07-87bd-0223ebedc6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:34.071177 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca203a-bdfd-4c07-87bd-0223ebedc6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:34.071564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca203a-bdfd-4c07-87bd-0223ebedc6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:34.071760 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca203a-bdfd-4c07-87bd-0223ebedc6b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:34.072492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca203a-bdfd-4c07-87bd-0223ebedc6b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:34.074110 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca203a-bdfd-4c07-87bd-0223ebedc6b5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:34.085674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1325/2647] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:34.090770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca203a-bdfd-4c07-87bd-0223ebedc6b5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:34.102813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca203a-bdfd-4c07-87bd-0223ebedc6b5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:34.109618 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d4b4f-a00c-4094-82f3-5b3896ca723e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:34.110817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d4b4f-a00c-4094-82f3-5b3896ca723e admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:34.111210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d4b4f-a00c-4094-82f3-5b3896ca723e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:34.111860 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d4b4f-a00c-4094-82f3-5b3896ca723e admin admin] PATH_INFO: `/v3/projects/fa795b3a57dd44c0bfadb83ff41e7123/users/41a223ac2248427e925eaf9b01fbf103/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:34.117746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1323/2648] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:34.129237 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0467f-4412-4b4a-96cb-e8b99f84eccb None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:34.130826 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d7f0467f-4412-4b4a-96cb-e8b99f84eccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:34.143768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d4b4f-a00c-4094-82f3-5b3896ca723e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa795b3a57dd44c0bfadb83ff41e7123, user_id=41a223ac2248427e925eaf9b01fbf103, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:34.147794 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d4b4f-a00c-4094-82f3-5b3896ca723e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:34.165884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1326/2649] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:34 2022] PUT /identity/v3/projects/fa795b3a57dd44c0bfadb83ff41e7123/users/41a223ac2248427e925eaf9b01fbf103/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:34.171150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d4b4f-a00c-4094-82f3-5b3896ca723e admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:34.211229 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0467f-4412-4b4a-96cb-e8b99f84eccb None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:34.266884 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:34.274624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d4b4f-a00c-4094-82f3-5b3896ca723e admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:34.283812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79752dc0-2d1c-45c4-97e0-fd86568ce353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:34.284574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752dc0-2d1c-45c4-97e0-fd86568ce353 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:34.285054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752dc0-2d1c-45c4-97e0-fd86568ce353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:34.285333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752dc0-2d1c-45c4-97e0-fd86568ce353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:34.286136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79752dc0-2d1c-45c4-97e0-fd86568ce353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:34.287673 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef58332-5006-4944-907c-306f73a7a645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:34.288027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79752dc0-2d1c-45c4-97e0-fd86568ce353 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:34.288456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58332-5006-4944-907c-306f73a7a645 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:34.288935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58332-5006-4944-907c-306f73a7a645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:34.289340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58332-5006-4944-907c-306f73a7a645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:34.300072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1327/2650] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:34.304111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79752dc0-2d1c-45c4-97e0-fd86568ce353 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:34.314548 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79752dc0-2d1c-45c4-97e0-fd86568ce353 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:34.320415 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff03ec4-d72b-43a2-9b17-dad49a605bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:34.321335 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff03ec4-d72b-43a2-9b17-dad49a605bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:34.321615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff03ec4-d72b-43a2-9b17-dad49a605bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:34.321889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff03ec4-d72b-43a2-9b17-dad49a605bc5 admin admin] PATH_INFO: `/v3/projects/fa795b3a57dd44c0bfadb83ff41e7123/users/41a223ac2248427e925eaf9b01fbf103/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:34.324404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff03ec4-d72b-43a2-9b17-dad49a605bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa795b3a57dd44c0bfadb83ff41e7123, user_id=41a223ac2248427e925eaf9b01fbf103, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:34.327698 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff03ec4-d72b-43a2-9b17-dad49a605bc5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:34.339433 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1328/2651] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:34 2022] PUT /identity/v3/projects/fa795b3a57dd44c0bfadb83ff41e7123/users/41a223ac2248427e925eaf9b01fbf103/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:34.350125 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb53e7-2ded-4b2e-a00c-f552caf3865f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:34.350519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb53e7-2ded-4b2e-a00c-f552caf3865f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:34.350829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb53e7-2ded-4b2e-a00c-f552caf3865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:34.369898 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef58332-5006-4944-907c-306f73a7a645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:34.371387 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef58332-5006-4944-907c-306f73a7a645 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:34.379463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-95fb53e7-2ded-4b2e-a00c-f552caf3865f None None] MFA Rules not processed for user `41a223ac2248427e925eaf9b01fbf103`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:34.380899 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-95fb53e7-2ded-4b2e-a00c-f552caf3865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:34.382325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1324/2652] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:34:34.476568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:34.483602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1329/2653] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:34 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:34.488843 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef58332-5006-4944-907c-306f73a7a645 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:34.498690 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef58332-5006-4944-907c-306f73a7a645 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:34.514577 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb53e7-2ded-4b2e-a00c-f552caf3865f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:34.516379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-95fb53e7-2ded-4b2e-a00c-f552caf3865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:34.568153 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739d2a8-9a1e-45ac-af14-4c0fa82ae605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:34.568850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739d2a8-9a1e-45ac-af14-4c0fa82ae605 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:34.569230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739d2a8-9a1e-45ac-af14-4c0fa82ae605 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:34.569506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739d2a8-9a1e-45ac-af14-4c0fa82ae605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:34.595885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb53e7-2ded-4b2e-a00c-f552caf3865f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:34.599721 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff80c0-0f19-469f-bc18-ba72ad061b40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:34.600379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff80c0-0f19-469f-bc18-ba72ad061b40 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:34.600655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff80c0-0f19-469f-bc18-ba72ad061b40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:34.600932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff80c0-0f19-469f-bc18-ba72ad061b40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:34.610457 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff80c0-0f19-469f-bc18-ba72ad061b40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:34.611949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff80c0-0f19-469f-bc18-ba72ad061b40 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:34.623091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1330/2654] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:34 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:34:34.635810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739d2a8-9a1e-45ac-af14-4c0fa82ae605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:34.638597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739d2a8-9a1e-45ac-af14-4c0fa82ae605 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:34.654020 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1325/2655] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:34.732868 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff80c0-0f19-469f-bc18-ba72ad061b40 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:34.741786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff80c0-0f19-469f-bc18-ba72ad061b40 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:34.745908 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce017c7-773d-4e4a-8fa8-8618147f8414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:34.746587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce017c7-773d-4e4a-8fa8-8618147f8414 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:34.746878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce017c7-773d-4e4a-8fa8-8618147f8414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:34.747644 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce017c7-773d-4e4a-8fa8-8618147f8414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:34.817005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce017c7-773d-4e4a-8fa8-8618147f8414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:34.819124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce017c7-773d-4e4a-8fa8-8618147f8414 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:34.830434 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1331/2656] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:44.646325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=81 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:44.825040 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=54 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:46.791247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2583d-7302-4f8d-ae4a-0ea6f804a1d8 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:46.791832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2583d-7302-4f8d-ae4a-0ea6f804a1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:46.791832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2583d-7302-4f8d-ae4a-0ea6f804a1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:46.812200 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-95b2583d-7302-4f8d-ae4a-0ea6f804a1d8 None None] MFA Rules not processed for user `5b82f76d685041d084dd5f332a5b1105`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:46.813456 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-95b2583d-7302-4f8d-ae4a-0ea6f804a1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:46.886758 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1326/2657] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:46.892199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce017c7-773d-4e4a-8fa8-8618147f8414 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:46.972764 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce017c7-773d-4e4a-8fa8-8618147f8414 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:46.977999 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35362d67-d1a7-4b83-b499-c814dc5b732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:46.979000 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35362d67-d1a7-4b83-b499-c814dc5b732d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:46.979356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35362d67-d1a7-4b83-b499-c814dc5b732d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:46.979801 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35362d67-d1a7-4b83-b499-c814dc5b732d admin admin] PATH_INFO: `/v3/users/eebc0d4cf563484a8cec9b0ddad10937` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:46.981216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35362d67-d1a7-4b83-b499-c814dc5b732d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eebc0d4cf563484a8cec9b0ddad10937)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:46.983980 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35362d67-d1a7-4b83-b499-c814dc5b732d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:47.035850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-35362d67-d1a7-4b83-b499-c814dc5b732d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eebc0d4cf563484a8cec9b0ddad10937'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.052768 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-35362d67-d1a7-4b83-b499-c814dc5b732d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eebc0d4cf563484a8cec9b0ddad10937'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.053500 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-35362d67-d1a7-4b83-b499-c814dc5b732d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eebc0d4cf563484a8cec9b0ddad10937'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.062824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2583d-7302-4f8d-ae4a-0ea6f804a1d8 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:47.064504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-35362d67-d1a7-4b83-b499-c814dc5b732d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eebc0d4cf563484a8cec9b0ddad10937'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.065222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-35362d67-d1a7-4b83-b499-c814dc5b732d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eebc0d4cf563484a8cec9b0ddad10937'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.077374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2583d-7302-4f8d-ae4a-0ea6f804a1d8 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:47.082556 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b8394-ce80-48ac-8080-2a9dece8de3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:47.083490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b8394-ce80-48ac-8080-2a9dece8de3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:47.083972 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b8394-ce80-48ac-8080-2a9dece8de3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:47.084252 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b8394-ce80-48ac-8080-2a9dece8de3a admin admin] PATH_INFO: `/v3/users/bf06847e997d4bc8bda98d9aece6b487` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:47.085503 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b8394-ce80-48ac-8080-2a9dece8de3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf06847e997d4bc8bda98d9aece6b487)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:47.089580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b8394-ce80-48ac-8080-2a9dece8de3a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:47.090398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1332/2658] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:46 2022] DELETE /identity/v3/users/eebc0d4cf563484a8cec9b0ddad10937 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:47.096051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318f6f4-2557-457e-a212-62b3585a8ced None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:47.096317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318f6f4-2557-457e-a212-62b3585a8ced None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:47.096628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318f6f4-2557-457e-a212-62b3585a8ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:47.117495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-5318f6f4-2557-457e-a212-62b3585a8ced None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:47.118673 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5318f6f4-2557-457e-a212-62b3585a8ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:47.156746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ec3b8394-ce80-48ac-8080-2a9dece8de3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf06847e997d4bc8bda98d9aece6b487'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.169852 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ec3b8394-ce80-48ac-8080-2a9dece8de3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf06847e997d4bc8bda98d9aece6b487'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.170657 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ec3b8394-ce80-48ac-8080-2a9dece8de3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf06847e997d4bc8bda98d9aece6b487'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.181180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ec3b8394-ce80-48ac-8080-2a9dece8de3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf06847e997d4bc8bda98d9aece6b487'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.182104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ec3b8394-ce80-48ac-8080-2a9dece8de3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf06847e997d4bc8bda98d9aece6b487'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.191404 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1333/2659] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:47.200070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1327/2660] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:47 2022] DELETE /identity/v3/users/bf06847e997d4bc8bda98d9aece6b487 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:34:47.201243 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318f6f4-2557-457e-a212-62b3585a8ced None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:47.203319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-5318f6f4-2557-457e-a212-62b3585a8ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:47.205124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdf2b1-fc91-4aef-931b-638b03e0e80a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:47.205421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdf2b1-fc91-4aef-931b-638b03e0e80a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:47.205701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdf2b1-fc91-4aef-931b-638b03e0e80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:47.224630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-27bdf2b1-fc91-4aef-931b-638b03e0e80a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:47.225862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-27bdf2b1-fc91-4aef-931b-638b03e0e80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:47.230812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:47.273155 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318f6f4-2557-457e-a212-62b3585a8ced None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:47.299919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1328/2661] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:34:47.313570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdf2b1-fc91-4aef-931b-638b03e0e80a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:47.315706 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-27bdf2b1-fc91-4aef-931b-638b03e0e80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:47.329779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1085b-ecb4-429e-9b2d-71797c4bbc7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:47.330576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1085b-ecb4-429e-9b2d-71797c4bbc7b service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:47.331049 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1085b-ecb4-429e-9b2d-71797c4bbc7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:47.331498 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1085b-ecb4-429e-9b2d-71797c4bbc7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:47.342089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1085b-ecb4-429e-9b2d-71797c4bbc7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:47.343735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1085b-ecb4-429e-9b2d-71797c4bbc7b service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:47.351621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:47.356070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1334/2662] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:47.429247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdf2b1-fc91-4aef-931b-638b03e0e80a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:47.434720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08ac33-d324-45fe-94d3-7d022d11e902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:47.435880 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08ac33-d324-45fe-94d3-7d022d11e902 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:47.437133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08ac33-d324-45fe-94d3-7d022d11e902 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:47.437651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08ac33-d324-45fe-94d3-7d022d11e902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:47.449543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08ac33-d324-45fe-94d3-7d022d11e902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:47.451401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08ac33-d324-45fe-94d3-7d022d11e902 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:47.464819 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1329/2663] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:34:47.832478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1085b-ecb4-429e-9b2d-71797c4bbc7b service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:47.836489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e9e1085b-ecb4-429e-9b2d-71797c4bbc7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:47.860984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1085b-ecb4-429e-9b2d-71797c4bbc7b service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:47.868496 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:47.869992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:47.870314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:47.870727 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] PATH_INFO: `/v3/projects/04f6d0b781f549a8a3fab77c43d0e274` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:47.872865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f6d0b781f549a8a3fab77c43d0e274)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:47.875216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:47.954465 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f6d0b781f549a8a3fab77c43d0e274', 'request_id': 'req-0cf8d1d2-84e3-435b-b1f7-79975c23f699'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.962437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f6d0b781f549a8a3fab77c43d0e274', 'request_id': 'req-0cf8d1d2-84e3-435b-b1f7-79975c23f699'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.975215 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f6d0b781f549a8a3fab77c43d0e274', 'request_id': 'req-0cf8d1d2-84e3-435b-b1f7-79975c23f699'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.995123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] The token cache is being invalidate because project 04f6d0b781f549a8a3fab77c43d0e274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:47.995729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.996308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:47.998275 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1335/2664] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:47 2022] DELETE /identity/v3/projects/04f6d0b781f549a8a3fab77c43d0e274 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:48.003174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08ac33-d324-45fe-94d3-7d022d11e902 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:48.005727 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:48.007972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:48.014100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08ac33-d324-45fe-94d3-7d022d11e902 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:48.091287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:48.092185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:48.093146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:48.093579 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] PATH_INFO: `/v3/users/59692774954b4074a62a7cd4e666451a` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:48.094862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59692774954b4074a62a7cd4e666451a)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:48.096925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:48.115403 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8d1d2-84e3-435b-b1f7-79975c23f699 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:48.122372 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:48.123302 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:48.123722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:48.124026 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] PATH_INFO: `/v3/projects/b68656329cd44267a500f6b0eb75b182` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:48.125307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68656329cd44267a500f6b0eb75b182)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:48.127307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:48.130478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:48.163665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59692774954b4074a62a7cd4e666451a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.183468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59692774954b4074a62a7cd4e666451a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.184463 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59692774954b4074a62a7cd4e666451a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.192689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68656329cd44267a500f6b0eb75b182', 'request_id': 'req-73569cfe-b167-4375-a7fa-0efc199f2b31'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.196533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59692774954b4074a62a7cd4e666451a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.198205 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59692774954b4074a62a7cd4e666451a'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.209328 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68656329cd44267a500f6b0eb75b182', 'request_id': 'req-73569cfe-b167-4375-a7fa-0efc199f2b31'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.224939 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68656329cd44267a500f6b0eb75b182', 'request_id': 'req-73569cfe-b167-4375-a7fa-0efc199f2b31'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.235961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1330/2665] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:48 2022] DELETE /identity/v3/users/59692774954b4074a62a7cd4e666451a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:48.256406 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] The token cache is being invalidate because project b68656329cd44267a500f6b0eb75b182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:48.256728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.258365 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.262789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1336/2666] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:48 2022] DELETE /identity/v3/projects/b68656329cd44267a500f6b0eb75b182 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:48.266781 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:48.269170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:48.363965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:48.417037 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db1f3ce-21f8-4d63-821d-b7b4012679b9 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:48.421998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62205fb8-96d9-4cc5-a615-363981057a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:48.423227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62205fb8-96d9-4cc5-a615-363981057a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:48.423378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62205fb8-96d9-4cc5-a615-363981057a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:48.423666 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62205fb8-96d9-4cc5-a615-363981057a18 admin admin] PATH_INFO: `/v3/users/41a223ac2248427e925eaf9b01fbf103` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:48.425129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62205fb8-96d9-4cc5-a615-363981057a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a223ac2248427e925eaf9b01fbf103)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:48.427300 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62205fb8-96d9-4cc5-a615-363981057a18 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:48.494985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-62205fb8-96d9-4cc5-a615-363981057a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a223ac2248427e925eaf9b01fbf103'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.508348 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-62205fb8-96d9-4cc5-a615-363981057a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a223ac2248427e925eaf9b01fbf103'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.509122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-62205fb8-96d9-4cc5-a615-363981057a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a223ac2248427e925eaf9b01fbf103'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.517269 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-62205fb8-96d9-4cc5-a615-363981057a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a223ac2248427e925eaf9b01fbf103'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.517986 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-62205fb8-96d9-4cc5-a615-363981057a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a223ac2248427e925eaf9b01fbf103'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:48.534518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1331/2667] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:34:48 2022] DELETE /identity/v3/users/41a223ac2248427e925eaf9b01fbf103 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:48.805926 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:48.807606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:48.966136 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73569cfe-b167-4375-a7fa-0efc199f2b31 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:49.038083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:49.039106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:49.039352 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:49.039605 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] PATH_INFO: `/v3/projects/05f33a73e6b54cda8747d522a061577d` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:49.041229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] RBAC: Authorizing `identity:delete_project(project_id=05f33a73e6b54cda8747d522a061577d)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:49.043244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:49.097745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:49.106193 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05f33a73e6b54cda8747d522a061577d', 'request_id': 'req-46951b59-5f4f-47e5-bea0-6f223fc3e4db'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:49.115726 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05f33a73e6b54cda8747d522a061577d', 'request_id': 'req-46951b59-5f4f-47e5-bea0-6f223fc3e4db'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:49.134245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05f33a73e6b54cda8747d522a061577d', 'request_id': 'req-46951b59-5f4f-47e5-bea0-6f223fc3e4db'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:49.153863 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] The token cache is being invalidate because project 05f33a73e6b54cda8747d522a061577d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:49.155156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:49.157560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:49.164045 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1337/2668] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:48 2022] DELETE /identity/v3/projects/05f33a73e6b54cda8747d522a061577d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:49.167225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea60e4-41ec-4675-95b1-0f97ee71f935 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:49.167817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea60e4-41ec-4675-95b1-0f97ee71f935 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:49.168016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea60e4-41ec-4675-95b1-0f97ee71f935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:49.210881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-afea60e4-41ec-4675-95b1-0f97ee71f935 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:49.212286 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-afea60e4-41ec-4675-95b1-0f97ee71f935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:49.237718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:49.240597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:49.378403 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1332/2669] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:49.393070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fce8f7-803b-4afe-a5c5-658a0e5a2f4a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:49.393381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fce8f7-803b-4afe-a5c5-658a0e5a2f4a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:49.394109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fce8f7-803b-4afe-a5c5-658a0e5a2f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:49.445578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-89fce8f7-803b-4afe-a5c5-658a0e5a2f4a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:49.447204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-89fce8f7-803b-4afe-a5c5-658a0e5a2f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:49.461663 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46951b59-5f4f-47e5-bea0-6f223fc3e4db admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:49.465886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1333/2670] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:49.468693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fce8f7-803b-4afe-a5c5-658a0e5a2f4a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:49.472981 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:49.474275 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:49.474921 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:49.475409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] PATH_INFO: `/v3/projects/fa795b3a57dd44c0bfadb83ff41e7123` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:49.478287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa795b3a57dd44c0bfadb83ff41e7123)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:49.480948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:49.485868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fce8f7-803b-4afe-a5c5-658a0e5a2f4a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:49.506000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e439d54-8db5-4397-b220-c83f75c56bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:49.506724 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e439d54-8db5-4397-b220-c83f75c56bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:49.507010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e439d54-8db5-4397-b220-c83f75c56bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:49.507287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e439d54-8db5-4397-b220-c83f75c56bcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:49.508198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e439d54-8db5-4397-b220-c83f75c56bcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:49.508516 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e439d54-8db5-4397-b220-c83f75c56bcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:49.510168 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e439d54-8db5-4397-b220-c83f75c56bcb admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:49.525934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1334/2671] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:34:49.530969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e439d54-8db5-4397-b220-c83f75c56bcb admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:49.547275 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e439d54-8db5-4397-b220-c83f75c56bcb admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:49.569697 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa795b3a57dd44c0bfadb83ff41e7123', 'request_id': 'req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:49.590035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa795b3a57dd44c0bfadb83ff41e7123', 'request_id': 'req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:49.606240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa795b3a57dd44c0bfadb83ff41e7123', 'request_id': 'req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:49.613921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:49.631963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] The token cache is being invalidate because project fa795b3a57dd44c0bfadb83ff41e7123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:34:49.632549 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:49.633668 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc2b3ecc-78af-4c41-9cec-018d2cba4f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:34:49.636569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1338/2672] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:34:49 2022] DELETE /identity/v3/projects/fa795b3a57dd44c0bfadb83ff41e7123 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:49.648884 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51813ddd-ed27-40b6-9eff-e1a6be96cbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:49.650292 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51813ddd-ed27-40b6-9eff-e1a6be96cbab admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:49.650630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51813ddd-ed27-40b6-9eff-e1a6be96cbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:49.650954 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51813ddd-ed27-40b6-9eff-e1a6be96cbab admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:49.655503 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51813ddd-ed27-40b6-9eff-e1a6be96cbab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:49.664779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51813ddd-ed27-40b6-9eff-e1a6be96cbab admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:49.711237 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0c13a-d5bc-4506-b678-ed53776467ff None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:49.712162 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0c13a-d5bc-4506-b678-ed53776467ff None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:49.712460 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0c13a-d5bc-4506-b678-ed53776467ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:49.730084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1335/2673] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:49 2022] POST /identity/v3/projects => generated 383 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:49.735072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51813ddd-ed27-40b6-9eff-e1a6be96cbab admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:49.752233 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b1a0c13a-d5bc-4506-b678-ed53776467ff None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:49.753964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b1a0c13a-d5bc-4506-b678-ed53776467ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:49.760322 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-51813ddd-ed27-40b6-9eff-e1a6be96cbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:49.916442 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:49.925824 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51813ddd-ed27-40b6-9eff-e1a6be96cbab admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:49.933016 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8ccd6-c4dc-4948-b33a-87a352ce98b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:49.934057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8ccd6-c4dc-4948-b33a-87a352ce98b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:49.934266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8ccd6-c4dc-4948-b33a-87a352ce98b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:49.934574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8ccd6-c4dc-4948-b33a-87a352ce98b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:49.936150 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8ccd6-c4dc-4948-b33a-87a352ce98b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:49.938823 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8ccd6-c4dc-4948-b33a-87a352ce98b2 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:49.954724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1339/2674] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:49.969775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf54d57-8fd4-4154-8b6a-56599cc765e8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:49.971624 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf54d57-8fd4-4154-8b6a-56599cc765e8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:49.973025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf54d57-8fd4-4154-8b6a-56599cc765e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:49.980783 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1336/2675] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:49 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:49.983536 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8ccd6-c4dc-4948-b33a-87a352ce98b2 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:50.003810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8ccd6-c4dc-4948-b33a-87a352ce98b2 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:50.018192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d645cf-f0a5-4d89-8eaf-22c2139a43f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:50.018988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d645cf-f0a5-4d89-8eaf-22c2139a43f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.019342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d645cf-f0a5-4d89-8eaf-22c2139a43f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.019840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d645cf-f0a5-4d89-8eaf-22c2139a43f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.020795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d645cf-f0a5-4d89-8eaf-22c2139a43f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:50.024202 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d645cf-f0a5-4d89-8eaf-22c2139a43f3 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:50.024646 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-dcf54d57-8fd4-4154-8b6a-56599cc765e8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:50.027687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-dcf54d57-8fd4-4154-8b6a-56599cc765e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:50.051447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1340/2676] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:50.052038 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1337/2677] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:34:50.058498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d645cf-f0a5-4d89-8eaf-22c2139a43f3 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:50.059246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf54d57-8fd4-4154-8b6a-56599cc765e8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:50.076027 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf54d57-8fd4-4154-8b6a-56599cc765e8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:50.077543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d645cf-f0a5-4d89-8eaf-22c2139a43f3 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:50.082845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7da060-1e58-4e6e-ad62-0b5146b2bcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:50.083537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7da060-1e58-4e6e-ad62-0b5146b2bcdd admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.083833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7da060-1e58-4e6e-ad62-0b5146b2bcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.084133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7da060-1e58-4e6e-ad62-0b5146b2bcdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.084972 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7da060-1e58-4e6e-ad62-0b5146b2bcdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:34:50.085264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7da060-1e58-4e6e-ad62-0b5146b2bcdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:50.087277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7da060-1e58-4e6e-ad62-0b5146b2bcdd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:50.093486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b2207-1f0c-4eba-af99-603b48a68f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:50.094354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b2207-1f0c-4eba-af99-603b48a68f16 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.094859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b2207-1f0c-4eba-af99-603b48a68f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.095139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b2207-1f0c-4eba-af99-603b48a68f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.095881 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b2207-1f0c-4eba-af99-603b48a68f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:50.104824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1341/2678] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:34:50.105022 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b2207-1f0c-4eba-af99-603b48a68f16 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:50.105858 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7da060-1e58-4e6e-ad62-0b5146b2bcdd admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:50.123433 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1338/2679] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:50.127325 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7da060-1e58-4e6e-ad62-0b5146b2bcdd admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:50.132709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43326f0-b611-410c-a68b-ae63194ac781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:50.133528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43326f0-b611-410c-a68b-ae63194ac781 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.133965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43326f0-b611-410c-a68b-ae63194ac781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.134338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43326f0-b611-410c-a68b-ae63194ac781 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.135295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43326f0-b611-410c-a68b-ae63194ac781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:50.138600 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43326f0-b611-410c-a68b-ae63194ac781 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:50.138797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b2207-1f0c-4eba-af99-603b48a68f16 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:50.164325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b2207-1f0c-4eba-af99-603b48a68f16 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:50.182383 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a6a04-42e9-491c-8593-eb65b8584029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:50.183105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1342/2680] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:34:50 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:34:50.183243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a6a04-42e9-491c-8593-eb65b8584029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.183243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a6a04-42e9-491c-8593-eb65b8584029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.183243 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a6a04-42e9-491c-8593-eb65b8584029 admin admin] PATH_INFO: `/v3/projects/daf2b77a77ba4e9481a417ed79a22b49/users/621ceb3079f74f16b93f2d78b64dbb29/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.184984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43326f0-b611-410c-a68b-ae63194ac781 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:50.225857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a6a04-42e9-491c-8593-eb65b8584029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf2b77a77ba4e9481a417ed79a22b49, user_id=621ceb3079f74f16b93f2d78b64dbb29, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:50.228837 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a6a04-42e9-491c-8593-eb65b8584029 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:50.280434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1339/2681] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:50 2022] PUT /identity/v3/projects/daf2b77a77ba4e9481a417ed79a22b49/users/621ceb3079f74f16b93f2d78b64dbb29/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:50.283899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f88c7-7d9b-4521-9499-e0906c1d5e88 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.284193 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f88c7-7d9b-4521-9499-e0906c1d5e88 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.284474 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f88c7-7d9b-4521-9499-e0906c1d5e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.293495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43326f0-b611-410c-a68b-ae63194ac781 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:50.318441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-868f88c7-7d9b-4521-9499-e0906c1d5e88 None None] MFA Rules not processed for user `621ceb3079f74f16b93f2d78b64dbb29`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:50.320917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-868f88c7-7d9b-4521-9499-e0906c1d5e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:50.432578 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:50.440473 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1340/2682] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:50.451608 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390f445-c550-4bf2-a04f-a65171e9545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:50.452927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390f445-c550-4bf2-a04f-a65171e9545c admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.453148 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390f445-c550-4bf2-a04f-a65171e9545c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.453420 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390f445-c550-4bf2-a04f-a65171e9545c admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.454111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390f445-c550-4bf2-a04f-a65171e9545c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:50.456075 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390f445-c550-4bf2-a04f-a65171e9545c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:50.459475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f88c7-7d9b-4521-9499-e0906c1d5e88 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:50.465169 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-868f88c7-7d9b-4521-9499-e0906c1d5e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:50.492371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1343/2683] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:34:50 2022] POST /identity/v3/users => generated 287 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:34:50.498072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390f445-c550-4bf2-a04f-a65171e9545c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:50.513305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390f445-c550-4bf2-a04f-a65171e9545c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:50.519927 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a256adb-c7c3-4337-bff9-5b25e0e1ee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:50.520800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a256adb-c7c3-4337-bff9-5b25e0e1ee2b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.521217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a256adb-c7c3-4337-bff9-5b25e0e1ee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.521810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a256adb-c7c3-4337-bff9-5b25e0e1ee2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.522343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a256adb-c7c3-4337-bff9-5b25e0e1ee2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:50.524346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a256adb-c7c3-4337-bff9-5b25e0e1ee2b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:50.541879 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1344/2684] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:50.544030 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a256adb-c7c3-4337-bff9-5b25e0e1ee2b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:50.562347 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a256adb-c7c3-4337-bff9-5b25e0e1ee2b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:50.569947 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b261e4-66d7-4635-8500-33f0b23bb439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:50.570854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b261e4-66d7-4635-8500-33f0b23bb439 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.571138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b261e4-66d7-4635-8500-33f0b23bb439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.571417 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b261e4-66d7-4635-8500-33f0b23bb439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.572601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b261e4-66d7-4635-8500-33f0b23bb439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:50.574405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b261e4-66d7-4635-8500-33f0b23bb439 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:50.592458 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1345/2685] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:34:50.596413 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b261e4-66d7-4635-8500-33f0b23bb439 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:50.598682 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f88c7-7d9b-4521-9499-e0906c1d5e88 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:50.606774 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9daeb-96d7-4268-827c-91eaab6b7b35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:50.607460 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9daeb-96d7-4268-827c-91eaab6b7b35 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.607749 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9daeb-96d7-4268-827c-91eaab6b7b35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.608064 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9daeb-96d7-4268-827c-91eaab6b7b35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.613146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b261e4-66d7-4635-8500-33f0b23bb439 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:50.619527 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ad125-1c49-4b1d-8433-699991177741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:50.620464 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ad125-1c49-4b1d-8433-699991177741 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.620876 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ad125-1c49-4b1d-8433-699991177741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.621156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ad125-1c49-4b1d-8433-699991177741 admin admin] PATH_INFO: `/v3/projects/bcb5131601b14b5884956a189e6e9462/users/4db856168c34491080b93c8ee5680c2a/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.629445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9daeb-96d7-4268-827c-91eaab6b7b35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:50.632997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae9daeb-96d7-4268-827c-91eaab6b7b35 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:50.644783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ad125-1c49-4b1d-8433-699991177741 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb5131601b14b5884956a189e6e9462, user_id=4db856168c34491080b93c8ee5680c2a, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:50.648625 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ad125-1c49-4b1d-8433-699991177741 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:50.662806 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1341/2686] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:50.697016 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1346/2687] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:34:50 2022] PUT /identity/v3/projects/bcb5131601b14b5884956a189e6e9462/users/4db856168c34491080b93c8ee5680c2a/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:34:50.699089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2763496-6c8c-46ca-8f72-7a9e62dc6747 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:50.699724 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2763496-6c8c-46ca-8f72-7a9e62dc6747 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:50.700225 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2763496-6c8c-46ca-8f72-7a9e62dc6747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:50.731453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c2763496-6c8c-46ca-8f72-7a9e62dc6747 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:50.732797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c2763496-6c8c-46ca-8f72-7a9e62dc6747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:50.883426 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1342/2688] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:50.896314 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ad125-1c49-4b1d-8433-699991177741 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:50.901848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1f8ad125-1c49-4b1d-8433-699991177741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:50.940201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:51.060589 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ad125-1c49-4b1d-8433-699991177741 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:51.075821 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ca261-d623-428a-85bc-908c6052e611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:51.076521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca261-d623-428a-85bc-908c6052e611 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:51.076963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca261-d623-428a-85bc-908c6052e611 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:51.077267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ca261-d623-428a-85bc-908c6052e611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:51.096861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ca261-d623-428a-85bc-908c6052e611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:51.098504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ca261-d623-428a-85bc-908c6052e611 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:51.119728 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1347/2689] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:51.537203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c59be-803c-44f7-9333-d67ecf4a4e11 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:51.537701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c59be-803c-44f7-9333-d67ecf4a4e11 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:51.538357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c59be-803c-44f7-9333-d67ecf4a4e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:51.572307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-069c59be-803c-44f7-9333-d67ecf4a4e11 None None] MFA Rules not processed for user `7d53405e3f88439f81b67106ad98adc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:51.575149 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-069c59be-803c-44f7-9333-d67ecf4a4e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:51.699754 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1343/2690] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:51.989234 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec43f7-3871-4029-b618-1baae4c41e08 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:51.989528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec43f7-3871-4029-b618-1baae4c41e08 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:51.989808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec43f7-3871-4029-b618-1baae4c41e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:52.030857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-b2ec43f7-3871-4029-b618-1baae4c41e08 None None] MFA Rules not processed for user `4db856168c34491080b93c8ee5680c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:52.032864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-b2ec43f7-3871-4029-b618-1baae4c41e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:52.117980 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1348/2691] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:34:52.129989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c59be-803c-44f7-9333-d67ecf4a4e11 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:34:52.131709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-069c59be-803c-44f7-9333-d67ecf4a4e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:52.196648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:34:52.207836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c59be-803c-44f7-9333-d67ecf4a4e11 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:34:52.218453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c0a3b-8b32-424f-94da-120e40b8f445 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:34:52.218753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c0a3b-8b32-424f-94da-120e40b8f445 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:52.219105 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c0a3b-8b32-424f-94da-120e40b8f445 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:52.219313 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c0a3b-8b32-424f-94da-120e40b8f445 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:52.243568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c0a3b-8b32-424f-94da-120e40b8f445 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:34:52.245961 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c0a3b-8b32-424f-94da-120e40b8f445 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:34:52.264073 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1344/2692] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:34:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:34:58.966519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c55569-07c0-49ab-88e6-7ba5ca3635a3 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:34:58.967032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c55569-07c0-49ab-88e6-7ba5ca3635a3 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:34:58.967253 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c55569-07c0-49ab-88e6-7ba5ca3635a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:34:58.990856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-62c55569-07c0-49ab-88e6-7ba5ca3635a3 None None] MFA Rules not processed for user `4db856168c34491080b93c8ee5680c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:34:58.992081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-62c55569-07c0-49ab-88e6-7ba5ca3635a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:34:59.001023 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1349/2693] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:34:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:01.164340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c0a3b-8b32-424f-94da-120e40b8f445 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:01.167315 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-538c0a3b-8b32-424f-94da-120e40b8f445 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:01.213179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c0a3b-8b32-424f-94da-120e40b8f445 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:01.221764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:01.222987 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:01.224809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:01.225352 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] PATH_INFO: `/v3/users/e3552aa56aa6467190ac54fb39cb7a82` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:01.227264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3552aa56aa6467190ac54fb39cb7a82)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:01.229804 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:01.330866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3552aa56aa6467190ac54fb39cb7a82'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:01.350650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3552aa56aa6467190ac54fb39cb7a82'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:01.352208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3552aa56aa6467190ac54fb39cb7a82'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:01.360555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3552aa56aa6467190ac54fb39cb7a82'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:01.361681 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3552aa56aa6467190ac54fb39cb7a82'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:01.393278 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1345/2694] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:01 2022] DELETE /identity/v3/users/e3552aa56aa6467190ac54fb39cb7a82 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:01.398298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d74ffa-2080-4370-af90-278521bff3f8 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:01.399395 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d74ffa-2080-4370-af90-278521bff3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:01.399925 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d74ffa-2080-4370-af90-278521bff3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:01.423903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-06d74ffa-2080-4370-af90-278521bff3f8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:01.426129 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-06d74ffa-2080-4370-af90-278521bff3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:01.554718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1350/2695] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:01.578762 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:01.580799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:01.714161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983a144-7dc8-44d8-8c8b-aeb895fa3d2c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:01.721582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd0ef8-d375-4835-a6a8-b781fa1b262d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:01.722250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd0ef8-d375-4835-a6a8-b781fa1b262d service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:01.722602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd0ef8-d375-4835-a6a8-b781fa1b262d service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:01.722877 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd0ef8-d375-4835-a6a8-b781fa1b262d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:01.743639 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd0ef8-d375-4835-a6a8-b781fa1b262d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:01.746379 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd0ef8-d375-4835-a6a8-b781fa1b262d service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:01.776075 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1346/2696] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:35:02.069190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d74ffa-2080-4370-af90-278521bff3f8 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:02.071526 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-06d74ffa-2080-4370-af90-278521bff3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:02.087595 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d74ffa-2080-4370-af90-278521bff3f8 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:02.092881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:02.094175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:02.095066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:02.095738 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] PATH_INFO: `/v3/users/621ceb3079f74f16b93f2d78b64dbb29` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:02.098312 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] RBAC: Authorizing `identity:delete_user(user_id=621ceb3079f74f16b93f2d78b64dbb29)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:02.100890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:02.144895 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:02.159229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '621ceb3079f74f16b93f2d78b64dbb29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.172998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '621ceb3079f74f16b93f2d78b64dbb29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.173769 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '621ceb3079f74f16b93f2d78b64dbb29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.183035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '621ceb3079f74f16b93f2d78b64dbb29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.183720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '621ceb3079f74f16b93f2d78b64dbb29'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.199339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1351/2697] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:02 2022] DELETE /identity/v3/users/621ceb3079f74f16b93f2d78b64dbb29 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:02.204130 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa052267-5b30-4936-863a-c0c80190d42c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:02.204650 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa052267-5b30-4936-863a-c0c80190d42c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:02.204887 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa052267-5b30-4936-863a-c0c80190d42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:02.221681 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-aa052267-5b30-4936-863a-c0c80190d42c None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:02.223041 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-aa052267-5b30-4936-863a-c0c80190d42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:02.302569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1347/2698] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:02.311369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:02.313127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:02.393953 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1385e7-1b8b-4ce3-b1d9-9a998384545c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:02.400976 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0939d-9ab6-4c61-b52e-73eff4fa0330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:02.402190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0939d-9ab6-4c61-b52e-73eff4fa0330 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:02.403396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0939d-9ab6-4c61-b52e-73eff4fa0330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:02.403734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c0939d-9ab6-4c61-b52e-73eff4fa0330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:02.423771 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0939d-9ab6-4c61-b52e-73eff4fa0330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:02.425295 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c0939d-9ab6-4c61-b52e-73eff4fa0330 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:02.430703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa052267-5b30-4936-863a-c0c80190d42c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:02.433715 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-aa052267-5b30-4936-863a-c0c80190d42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:02.437808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1352/2699] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:02.445267 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:02.451919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa052267-5b30-4936-863a-c0c80190d42c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:02.458865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:02.461409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:02.462432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:02.463102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] PATH_INFO: `/v3/projects/8a0935e619f143e58d185db5a8a581f1` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:02.464836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a0935e619f143e58d185db5a8a581f1)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:02.467314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:02.548216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a0935e619f143e58d185db5a8a581f1', 'request_id': 'req-954582aa-2970-44f9-9d2f-584caccc001e'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.556610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a0935e619f143e58d185db5a8a581f1', 'request_id': 'req-954582aa-2970-44f9-9d2f-584caccc001e'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.567184 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a0935e619f143e58d185db5a8a581f1', 'request_id': 'req-954582aa-2970-44f9-9d2f-584caccc001e'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.582034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] The token cache is being invalidate because project 8a0935e619f143e58d185db5a8a581f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:02.582324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.583054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.584932 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1348/2700] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:02 2022] DELETE /identity/v3/projects/8a0935e619f143e58d185db5a8a581f1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:35:02.589032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0939d-9ab6-4c61-b52e-73eff4fa0330 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:02.601145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c0939d-9ab6-4c61-b52e-73eff4fa0330 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:02.692929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046acb1d-b616-4161-908c-900d864b6cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:02.693854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046acb1d-b616-4161-908c-900d864b6cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:02.694208 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046acb1d-b616-4161-908c-900d864b6cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:02.694729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046acb1d-b616-4161-908c-900d864b6cfa admin admin] PATH_INFO: `/v3/users/1b9aa67988ca414993a9dcc485afeab6` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:02.696280 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046acb1d-b616-4161-908c-900d864b6cfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9aa67988ca414993a9dcc485afeab6)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:02.698216 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046acb1d-b616-4161-908c-900d864b6cfa admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:02.763197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-046acb1d-b616-4161-908c-900d864b6cfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9aa67988ca414993a9dcc485afeab6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.770275 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:02.780556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-046acb1d-b616-4161-908c-900d864b6cfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9aa67988ca414993a9dcc485afeab6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.781877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-046acb1d-b616-4161-908c-900d864b6cfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9aa67988ca414993a9dcc485afeab6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.791150 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-046acb1d-b616-4161-908c-900d864b6cfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9aa67988ca414993a9dcc485afeab6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.791865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-046acb1d-b616-4161-908c-900d864b6cfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9aa67988ca414993a9dcc485afeab6'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:02.809436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1353/2701] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:02 2022] DELETE /identity/v3/users/1b9aa67988ca414993a9dcc485afeab6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:03.536575 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:03.538082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:03.564172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954582aa-2970-44f9-9d2f-584caccc001e admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:03.674664 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:03.676038 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:03.676381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:03.676766 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] PATH_INFO: `/v3/projects/daf2b77a77ba4e9481a417ed79a22b49` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:03.680427 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=daf2b77a77ba4e9481a417ed79a22b49)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:03.684964 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:03.757882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daf2b77a77ba4e9481a417ed79a22b49', 'request_id': 'req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:03.769135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daf2b77a77ba4e9481a417ed79a22b49', 'request_id': 'req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:03.781342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daf2b77a77ba4e9481a417ed79a22b49', 'request_id': 'req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:03.801729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] The token cache is being invalidate because project daf2b77a77ba4e9481a417ed79a22b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:03.802068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:03.802835 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e4929f62-4ec7-44b3-8edb-c3f05bcd3ebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:03.804988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1349/2702] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:03 2022] DELETE /identity/v3/projects/daf2b77a77ba4e9481a417ed79a22b49 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:35:03.810105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c21e4e-4d70-45c0-9e9c-1de82c201561 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:03.810602 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c21e4e-4d70-45c0-9e9c-1de82c201561 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:03.810735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c21e4e-4d70-45c0-9e9c-1de82c201561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:03.842244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e7c21e4e-4d70-45c0-9e9c-1de82c201561 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:03.843575 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e7c21e4e-4d70-45c0-9e9c-1de82c201561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:03.927198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1354/2703] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:03.936697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaadcd-5553-4196-953a-f08512c26474 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:03.936982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaadcd-5553-4196-953a-f08512c26474 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:03.937260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaadcd-5553-4196-953a-f08512c26474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:03.963299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-29aaadcd-5553-4196-953a-f08512c26474 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:03.964576 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-29aaadcd-5553-4196-953a-f08512c26474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:03.973942 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1350/2704] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:03.979066 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c21e4e-4d70-45c0-9e9c-1de82c201561 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:03.991322 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c21e4e-4d70-45c0-9e9c-1de82c201561 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:03.996431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154769ed-f086-48b1-9bdb-00c4d992f031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:03.997076 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154769ed-f086-48b1-9bdb-00c4d992f031 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:03.997363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154769ed-f086-48b1-9bdb-00c4d992f031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:03.997643 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154769ed-f086-48b1-9bdb-00c4d992f031 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:03.998387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154769ed-f086-48b1-9bdb-00c4d992f031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:35:03.999242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154769ed-f086-48b1-9bdb-00c4d992f031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.000964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154769ed-f086-48b1-9bdb-00c4d992f031 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.017185 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1355/2705] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:35:04.021758 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaadcd-5553-4196-953a-f08512c26474 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.031836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaadcd-5553-4196-953a-f08512c26474 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.036745 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce8377-3f2f-4ceb-a105-567c79b5c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.037436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce8377-3f2f-4ceb-a105-567c79b5c71e admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.037722 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce8377-3f2f-4ceb-a105-567c79b5c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.037998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce8377-3f2f-4ceb-a105-567c79b5c71e admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.038741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce8377-3f2f-4ceb-a105-567c79b5c71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.040737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce8377-3f2f-4ceb-a105-567c79b5c71e admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.052973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:04.088504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1351/2706] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:35:04 2022] POST /identity/v3/projects => generated 359 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:04.089973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154769ed-f086-48b1-9bdb-00c4d992f031 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.100542 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154769ed-f086-48b1-9bdb-00c4d992f031 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.173574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8f008-1e43-49a5-8484-4622e4909e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.174327 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8f008-1e43-49a5-8484-4622e4909e11 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.174690 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8f008-1e43-49a5-8484-4622e4909e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.175019 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8f008-1e43-49a5-8484-4622e4909e11 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.176528 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8f008-1e43-49a5-8484-4622e4909e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.179436 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8f008-1e43-49a5-8484-4622e4909e11 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.207708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1356/2707] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:35:04 2022] POST /identity/v3/users => generated 280 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:04.215984 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce8377-3f2f-4ceb-a105-567c79b5c71e admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.227152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce8377-3f2f-4ceb-a105-567c79b5c71e admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.235270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd69618-3012-47b7-93c6-72ef0acdfb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.235978 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd69618-3012-47b7-93c6-72ef0acdfb07 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.236260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd69618-3012-47b7-93c6-72ef0acdfb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.236539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd69618-3012-47b7-93c6-72ef0acdfb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.237364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd69618-3012-47b7-93c6-72ef0acdfb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.239104 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd69618-3012-47b7-93c6-72ef0acdfb07 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.251452 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1352/2708] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:35:04.255729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8f008-1e43-49a5-8484-4622e4909e11 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.266890 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8f008-1e43-49a5-8484-4622e4909e11 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.276785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74437241-bc1a-416a-a12c-393c9da3058b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.277401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74437241-bc1a-416a-a12c-393c9da3058b admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.278033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74437241-bc1a-416a-a12c-393c9da3058b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.278033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74437241-bc1a-416a-a12c-393c9da3058b admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.278718 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74437241-bc1a-416a-a12c-393c9da3058b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.280375 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74437241-bc1a-416a-a12c-393c9da3058b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.282921 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd69618-3012-47b7-93c6-72ef0acdfb07 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.284772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3dd69618-3012-47b7-93c6-72ef0acdfb07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:04.296588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1357/2709] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:04.301685 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74437241-bc1a-416a-a12c-393c9da3058b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.309924 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd69618-3012-47b7-93c6-72ef0acdfb07 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.313050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74437241-bc1a-416a-a12c-393c9da3058b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.317880 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5515c5d-11ce-45cd-b623-9a3c26118f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.318162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.318829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5515c5d-11ce-45cd-b623-9a3c26118f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.319078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.319370 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.319645 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] PATH_INFO: `/v3/projects/265dcad1c93b40d58c4e0128efc523f8` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.319773 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5515c5d-11ce-45cd-b623-9a3c26118f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.319993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5515c5d-11ce-45cd-b623-9a3c26118f55 admin admin] PATH_INFO: `/v3/projects/7921e14b496448c5bd63b7586b2112d3/users/985fdffb94f045938dc1271be5dda805/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.320908 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=265dcad1c93b40d58c4e0128efc523f8)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.323135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.338724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5515c5d-11ce-45cd-b623-9a3c26118f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7921e14b496448c5bd63b7586b2112d3, user_id=985fdffb94f045938dc1271be5dda805, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.341438 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5515c5d-11ce-45cd-b623-9a3c26118f55 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.358573 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1358/2710] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:04 2022] PUT /identity/v3/projects/7921e14b496448c5bd63b7586b2112d3/users/985fdffb94f045938dc1271be5dda805/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:04.369166 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bea334-0c26-47a6-8780-8454ff1108ce None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.369821 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bea334-0c26-47a6-8780-8454ff1108ce None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.370106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bea334-0c26-47a6-8780-8454ff1108ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.376413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '265dcad1c93b40d58c4e0128efc523f8', 'request_id': 'req-cab6c20b-8f1b-4388-9110-867022bfe031'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:04.391502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:04.392199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '265dcad1c93b40d58c4e0128efc523f8', 'request_id': 'req-cab6c20b-8f1b-4388-9110-867022bfe031'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:04.401962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '265dcad1c93b40d58c4e0128efc523f8', 'request_id': 'req-cab6c20b-8f1b-4388-9110-867022bfe031'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:04.408948 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-38bea334-0c26-47a6-8780-8454ff1108ce None None] MFA Rules not processed for user `985fdffb94f045938dc1271be5dda805`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:04.410469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-38bea334-0c26-47a6-8780-8454ff1108ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:04.421938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] The token cache is being invalidate because project 265dcad1c93b40d58c4e0128efc523f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:04.422355 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:04.423542 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-cab6c20b-8f1b-4388-9110-867022bfe031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:04.425686 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1353/2711] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:04 2022] DELETE /identity/v3/projects/265dcad1c93b40d58c4e0128efc523f8 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:04.440344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6830f1-8550-4b73-90d4-459680ca6bf7 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.440344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6830f1-8550-4b73-90d4-459680ca6bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.440752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6830f1-8550-4b73-90d4-459680ca6bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.462389 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-aa6830f1-8550-4b73-90d4-459680ca6bf7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:04.463714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-aa6830f1-8550-4b73-90d4-459680ca6bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:04.509133 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1359/2712] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:04 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:04.541843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bea334-0c26-47a6-8780-8454ff1108ce None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.544273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-38bea334-0c26-47a6-8780-8454ff1108ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:04.547010 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1354/2713] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:04.555569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0141043-f55a-46b8-84e2-b7325e5091d8 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.555977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0141043-f55a-46b8-84e2-b7325e5091d8 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.556325 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0141043-f55a-46b8-84e2-b7325e5091d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.576917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c0141043-f55a-46b8-84e2-b7325e5091d8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:04.578868 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c0141043-f55a-46b8-84e2-b7325e5091d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:04.600720 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1355/2714] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:35:04.604817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0141043-f55a-46b8-84e2-b7325e5091d8 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.616279 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0141043-f55a-46b8-84e2-b7325e5091d8 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.626941 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169c69e-f9b9-4a3b-9333-e49ea293e916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.628509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169c69e-f9b9-4a3b-9333-e49ea293e916 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.628716 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bea334-0c26-47a6-8780-8454ff1108ce None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.628997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169c69e-f9b9-4a3b-9333-e49ea293e916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.629398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0169c69e-f9b9-4a3b-9333-e49ea293e916 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.630893 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169c69e-f9b9-4a3b-9333-e49ea293e916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:35:04.631428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169c69e-f9b9-4a3b-9333-e49ea293e916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.634159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0169c69e-f9b9-4a3b-9333-e49ea293e916 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.636588 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f4738-f52c-4e49-8bd9-bd7b74b060d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.637269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f4738-f52c-4e49-8bd9-bd7b74b060d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.637704 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f4738-f52c-4e49-8bd9-bd7b74b060d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.637991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f4738-f52c-4e49-8bd9-bd7b74b060d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.648511 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f4738-f52c-4e49-8bd9-bd7b74b060d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.649696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f4738-f52c-4e49-8bd9-bd7b74b060d1 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.654510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1356/2715] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:35:04.659354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169c69e-f9b9-4a3b-9333-e49ea293e916 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.666689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1360/2716] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:04 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:35:04.678889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0169c69e-f9b9-4a3b-9333-e49ea293e916 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.680594 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f4738-f52c-4e49-8bd9-bd7b74b060d1 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.682929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d20f4738-f52c-4e49-8bd9-bd7b74b060d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:04.684956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558232c-6ead-41ee-96eb-5ea5479c00ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.685737 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558232c-6ead-41ee-96eb-5ea5479c00ce admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.686162 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558232c-6ead-41ee-96eb-5ea5479c00ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.686461 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558232c-6ead-41ee-96eb-5ea5479c00ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.687240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558232c-6ead-41ee-96eb-5ea5479c00ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.689482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558232c-6ead-41ee-96eb-5ea5479c00ce admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.698448 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f4738-f52c-4e49-8bd9-bd7b74b060d1 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.703561 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2ce79-dbd2-4630-a6e0-f908189f3d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.704456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2ce79-dbd2-4630-a6e0-f908189f3d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.704730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2ce79-dbd2-4630-a6e0-f908189f3d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.705025 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2ce79-dbd2-4630-a6e0-f908189f3d93 admin admin] PATH_INFO: `/v3/users/985fdffb94f045938dc1271be5dda805` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.707010 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2ce79-dbd2-4630-a6e0-f908189f3d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=985fdffb94f045938dc1271be5dda805)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.709004 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2ce79-dbd2-4630-a6e0-f908189f3d93 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.731144 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1357/2717] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:35:04 2022] POST /identity/v3/projects => generated 363 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:35:04.736100 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558232c-6ead-41ee-96eb-5ea5479c00ce admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.761161 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:04.793967 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-31d2ce79-dbd2-4630-a6e0-f908189f3d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '985fdffb94f045938dc1271be5dda805'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:04.808637 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-31d2ce79-dbd2-4630-a6e0-f908189f3d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '985fdffb94f045938dc1271be5dda805'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:04.809409 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-31d2ce79-dbd2-4630-a6e0-f908189f3d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '985fdffb94f045938dc1271be5dda805'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:04.816394 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558232c-6ead-41ee-96eb-5ea5479c00ce admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.817709 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-31d2ce79-dbd2-4630-a6e0-f908189f3d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '985fdffb94f045938dc1271be5dda805'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:04.818628 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-31d2ce79-dbd2-4630-a6e0-f908189f3d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '985fdffb94f045938dc1271be5dda805'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:04.825638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada244a2-3a4c-40d2-8cdb-7f623424ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.826802 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada244a2-3a4c-40d2-8cdb-7f623424ddad admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.827274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada244a2-3a4c-40d2-8cdb-7f623424ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.827782 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada244a2-3a4c-40d2-8cdb-7f623424ddad admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.829122 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada244a2-3a4c-40d2-8cdb-7f623424ddad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.831129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada244a2-3a4c-40d2-8cdb-7f623424ddad admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.837606 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1361/2718] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:04 2022] DELETE /identity/v3/users/985fdffb94f045938dc1271be5dda805 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:04.843786 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c8a8b-cafa-437f-87e2-25767e8f7cd1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.844070 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c8a8b-cafa-437f-87e2-25767e8f7cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.844480 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c8a8b-cafa-437f-87e2-25767e8f7cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.855962 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1358/2719] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:35:04 2022] POST /identity/v3/users => generated 282 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:35:04.860587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada244a2-3a4c-40d2-8cdb-7f623424ddad admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.862266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ada244a2-3a4c-40d2-8cdb-7f623424ddad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:04.865509 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-411c8a8b-cafa-437f-87e2-25767e8f7cd1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:04.866756 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-411c8a8b-cafa-437f-87e2-25767e8f7cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:04.875272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada244a2-3a4c-40d2-8cdb-7f623424ddad admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:04.876809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:04.940964 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1362/2720] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:35:04.951168 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c8a8b-cafa-437f-87e2-25767e8f7cd1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:04.953392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-411c8a8b-cafa-437f-87e2-25767e8f7cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:04.977277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98018105-9428-4d38-a9c9-816bcdf4213c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:04.977965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98018105-9428-4d38-a9c9-816bcdf4213c admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:04.978258 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98018105-9428-4d38-a9c9-816bcdf4213c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:04.978678 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98018105-9428-4d38-a9c9-816bcdf4213c admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:04.979425 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98018105-9428-4d38-a9c9-816bcdf4213c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:04.981091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98018105-9428-4d38-a9c9-816bcdf4213c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:04.997621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1359/2721] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:35:05.004159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98018105-9428-4d38-a9c9-816bcdf4213c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:05.018574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98018105-9428-4d38-a9c9-816bcdf4213c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:05.026519 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb934dd5-e9e0-4e52-a24d-12e9e78f0b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:05.027608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb934dd5-e9e0-4e52-a24d-12e9e78f0b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.028062 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb934dd5-e9e0-4e52-a24d-12e9e78f0b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.028533 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb934dd5-e9e0-4e52-a24d-12e9e78f0b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.029815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb934dd5-e9e0-4e52-a24d-12e9e78f0b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:05.033198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb934dd5-e9e0-4e52-a24d-12e9e78f0b09 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:05.040266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c8a8b-cafa-437f-87e2-25767e8f7cd1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:05.046626 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc06e1-2d8e-4461-b6c4-4db2ac8b7179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:05.047836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc06e1-2d8e-4461-b6c4-4db2ac8b7179 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.048355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc06e1-2d8e-4461-b6c4-4db2ac8b7179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.048865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc06e1-2d8e-4461-b6c4-4db2ac8b7179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.054487 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1360/2722] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:05.056596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb934dd5-e9e0-4e52-a24d-12e9e78f0b09 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:05.065872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc06e1-2d8e-4461-b6c4-4db2ac8b7179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:05.067759 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc06e1-2d8e-4461-b6c4-4db2ac8b7179 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:05.074966 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb934dd5-e9e0-4e52-a24d-12e9e78f0b09 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:05.081660 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1363/2723] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:35:05.082123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302dd9c1-c786-43b8-98a7-fc2b2cb162a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:05.083157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302dd9c1-c786-43b8-98a7-fc2b2cb162a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.083499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302dd9c1-c786-43b8-98a7-fc2b2cb162a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.083844 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302dd9c1-c786-43b8-98a7-fc2b2cb162a8 admin admin] PATH_INFO: `/v3/projects/01e91a601fef4fe784b5e5e699b1d879/users/5b1bbfbd59c749f68955e76c1d462fbb/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.107464 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302dd9c1-c786-43b8-98a7-fc2b2cb162a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e91a601fef4fe784b5e5e699b1d879, user_id=5b1bbfbd59c749f68955e76c1d462fbb, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:05.111465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302dd9c1-c786-43b8-98a7-fc2b2cb162a8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:05.128675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1361/2724] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:05 2022] PUT /identity/v3/projects/01e91a601fef4fe784b5e5e699b1d879/users/5b1bbfbd59c749f68955e76c1d462fbb/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:05.141565 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ec3ee-e460-4709-8c5a-5fb9197fa53d None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.142006 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ec3ee-e460-4709-8c5a-5fb9197fa53d None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.142293 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ec3ee-e460-4709-8c5a-5fb9197fa53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.189182 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ce3ec3ee-e460-4709-8c5a-5fb9197fa53d None None] MFA Rules not processed for user `5b1bbfbd59c749f68955e76c1d462fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:05.190750 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ce3ec3ee-e460-4709-8c5a-5fb9197fa53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:05.269128 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1364/2725] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:05.274138 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302dd9c1-c786-43b8-98a7-fc2b2cb162a8 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:05.377297 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302dd9c1-c786-43b8-98a7-fc2b2cb162a8 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:05.384635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1894f3-a5ef-4937-81ac-e508e7606410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:05.385991 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1894f3-a5ef-4937-81ac-e508e7606410 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.386482 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1894f3-a5ef-4937-81ac-e508e7606410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.387000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1894f3-a5ef-4937-81ac-e508e7606410 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.388291 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1894f3-a5ef-4937-81ac-e508e7606410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:05.392842 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1894f3-a5ef-4937-81ac-e508e7606410 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:05.423157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:05.463000 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1362/2726] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:35:05 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:35:05.466437 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ec3ee-e460-4709-8c5a-5fb9197fa53d None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:05.483387 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ec3ee-e460-4709-8c5a-5fb9197fa53d None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:05.536412 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:05.552363 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898d993-9bca-4292-8b85-45c8e88e1f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:05.553540 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898d993-9bca-4292-8b85-45c8e88e1f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.554214 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898d993-9bca-4292-8b85-45c8e88e1f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.554762 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898d993-9bca-4292-8b85-45c8e88e1f62 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.555670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898d993-9bca-4292-8b85-45c8e88e1f62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:05.558315 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898d993-9bca-4292-8b85-45c8e88e1f62 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:05.596397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1365/2727] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:35:05 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:05.598507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1894f3-a5ef-4937-81ac-e508e7606410 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:05.606735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898d993-9bca-4292-8b85-45c8e88e1f62 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:05.609300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7898d993-9bca-4292-8b85-45c8e88e1f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:05.620851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1894f3-a5ef-4937-81ac-e508e7606410 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:05.629198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898d993-9bca-4292-8b85-45c8e88e1f62 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:05.629415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6ad4f-f40f-48a8-964b-6076de90b4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:05.630502 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6ad4f-f40f-48a8-964b-6076de90b4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.631311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6ad4f-f40f-48a8-964b-6076de90b4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.631900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6ad4f-f40f-48a8-964b-6076de90b4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.633398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6ad4f-f40f-48a8-964b-6076de90b4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:05.634381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:05.635308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.635659 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.636259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] PATH_INFO: `/v3/projects/7921e14b496448c5bd63b7586b2112d3` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.638371 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7921e14b496448c5bd63b7586b2112d3)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:05.639965 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:05.651423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6ad4f-f40f-48a8-964b-6076de90b4f2 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:05.676620 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1363/2728] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:05.680052 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6ad4f-f40f-48a8-964b-6076de90b4f2 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:05.699442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6ad4f-f40f-48a8-964b-6076de90b4f2 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:05.707842 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa40bf-7ab6-4fdf-a09e-b55521fff5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:05.708840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa40bf-7ab6-4fdf-a09e-b55521fff5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.709124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa40bf-7ab6-4fdf-a09e-b55521fff5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.709447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa40bf-7ab6-4fdf-a09e-b55521fff5e9 admin admin] PATH_INFO: `/v3/projects/2aff9f1bec0f474b83ecaa647d093877/users/553d913e82af452b86a0ba8a83ea4dee/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.727116 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa40bf-7ab6-4fdf-a09e-b55521fff5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aff9f1bec0f474b83ecaa647d093877, user_id=553d913e82af452b86a0ba8a83ea4dee, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:05.729910 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa40bf-7ab6-4fdf-a09e-b55521fff5e9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:05.744393 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7921e14b496448c5bd63b7586b2112d3', 'request_id': 'req-bc8eb63c-9453-4a1a-b311-631bac2721c0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:05.747923 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1364/2729] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:05 2022] PUT /identity/v3/projects/2aff9f1bec0f474b83ecaa647d093877/users/553d913e82af452b86a0ba8a83ea4dee/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:05.750014 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa40bf-7ab6-4fdf-a09e-b55521fff5e9 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:05.753831 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7921e14b496448c5bd63b7586b2112d3', 'request_id': 'req-bc8eb63c-9453-4a1a-b311-631bac2721c0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:05.762811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7921e14b496448c5bd63b7586b2112d3', 'request_id': 'req-bc8eb63c-9453-4a1a-b311-631bac2721c0'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:05.776701 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] The token cache is being invalidate because project 7921e14b496448c5bd63b7586b2112d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:05.776993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:05.777662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-bc8eb63c-9453-4a1a-b311-631bac2721c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:05.779631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1366/2730] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:05 2022] DELETE /identity/v3/projects/7921e14b496448c5bd63b7586b2112d3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:05.784787 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ba554-d993-4b43-93d9-6abcaaf21ddf None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.785072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ba554-d993-4b43-93d9-6abcaaf21ddf None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.785348 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ba554-d993-4b43-93d9-6abcaaf21ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.804651 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ba8ba554-d993-4b43-93d9-6abcaaf21ddf None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:05.806055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ba8ba554-d993-4b43-93d9-6abcaaf21ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:05.865595 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa40bf-7ab6-4fdf-a09e-b55521fff5e9 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:05.873186 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a54bce-8ffc-4e45-93ec-c288a526b28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:05.874915 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a54bce-8ffc-4e45-93ec-c288a526b28c admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.875307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a54bce-8ffc-4e45-93ec-c288a526b28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.875688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a54bce-8ffc-4e45-93ec-c288a526b28c admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.876888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a54bce-8ffc-4e45-93ec-c288a526b28c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:05.879555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a54bce-8ffc-4e45-93ec-c288a526b28c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:05.901239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1365/2731] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:05.902453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a54bce-8ffc-4e45-93ec-c288a526b28c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:05.918238 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b5a54bce-8ffc-4e45-93ec-c288a526b28c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:05.932804 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a54bce-8ffc-4e45-93ec-c288a526b28c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:05.939085 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86aced-0047-454d-8c2e-a83546066e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:05.939974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86aced-0047-454d-8c2e-a83546066e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.940254 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86aced-0047-454d-8c2e-a83546066e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.940530 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86aced-0047-454d-8c2e-a83546066e88 admin admin] PATH_INFO: `/v3/projects/2aff9f1bec0f474b83ecaa647d093877/users/553d913e82af452b86a0ba8a83ea4dee/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.942769 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86aced-0047-454d-8c2e-a83546066e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aff9f1bec0f474b83ecaa647d093877, user_id=553d913e82af452b86a0ba8a83ea4dee, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:05.945982 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86aced-0047-454d-8c2e-a83546066e88 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:05.946878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1367/2732] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:35:05.958957 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27215e4d-f05c-4f2e-87a4-fa15968e6fd1 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.959174 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1366/2733] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:05 2022] PUT /identity/v3/projects/2aff9f1bec0f474b83ecaa647d093877/users/553d913e82af452b86a0ba8a83ea4dee/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:05.959919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27215e4d-f05c-4f2e-87a4-fa15968e6fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.960453 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27215e4d-f05c-4f2e-87a4-fa15968e6fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.980499 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b017b-cfae-4996-9372-a8731b5ab2ca None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:05.981033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b017b-cfae-4996-9372-a8731b5ab2ca None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:05.981790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b017b-cfae-4996-9372-a8731b5ab2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:05.993194 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-27215e4d-f05c-4f2e-87a4-fa15968e6fd1 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:05.996300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-27215e4d-f05c-4f2e-87a4-fa15968e6fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:06.027314 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-aa6b017b-cfae-4996-9372-a8731b5ab2ca None None] MFA Rules not processed for user `553d913e82af452b86a0ba8a83ea4dee`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:06.028770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-aa6b017b-cfae-4996-9372-a8731b5ab2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:06.125912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1368/2734] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:06.132383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27215e4d-f05c-4f2e-87a4-fa15968e6fd1 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:06.139050 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:06.155827 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1367/2735] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:35:06.160082 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27215e4d-f05c-4f2e-87a4-fa15968e6fd1 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:06.168836 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba6285-67d2-443d-9ec3-79b017744bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:06.169727 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba6285-67d2-443d-9ec3-79b017744bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:06.170005 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba6285-67d2-443d-9ec3-79b017744bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:06.170358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba6285-67d2-443d-9ec3-79b017744bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:06.171503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba6285-67d2-443d-9ec3-79b017744bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:35:06.171934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba6285-67d2-443d-9ec3-79b017744bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:06.173830 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ba6285-67d2-443d-9ec3-79b017744bc5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:06.175221 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b017b-cfae-4996-9372-a8731b5ab2ca None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:06.177056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-aa6b017b-cfae-4996-9372-a8731b5ab2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:06.198576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1369/2736] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:35:06.203355 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba6285-67d2-443d-9ec3-79b017744bc5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:06.222828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba6285-67d2-443d-9ec3-79b017744bc5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:06.235379 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd16c7-9cd7-4cdf-a503-77a74d58e425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:06.236250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd16c7-9cd7-4cdf-a503-77a74d58e425 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:06.238325 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd16c7-9cd7-4cdf-a503-77a74d58e425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:06.238619 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:06.238857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd16c7-9cd7-4cdf-a503-77a74d58e425 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:06.241089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd16c7-9cd7-4cdf-a503-77a74d58e425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:06.244299 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cd16c7-9cd7-4cdf-a503-77a74d58e425 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:06.283794 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1370/2737] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:35:06 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:35:06.294707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd16c7-9cd7-4cdf-a503-77a74d58e425 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:06.309608 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b017b-cfae-4996-9372-a8731b5ab2ca None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:06.415459 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb681f5-780b-4644-a24c-58927ee6f329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:06.416454 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb681f5-780b-4644-a24c-58927ee6f329 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:06.416811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb681f5-780b-4644-a24c-58927ee6f329 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:06.417192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb681f5-780b-4644-a24c-58927ee6f329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:06.427392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd16c7-9cd7-4cdf-a503-77a74d58e425 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:06.436865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06083932-dfe8-4804-997d-bd89c920c7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:06.438047 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06083932-dfe8-4804-997d-bd89c920c7ae admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:06.438568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06083932-dfe8-4804-997d-bd89c920c7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:06.439072 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06083932-dfe8-4804-997d-bd89c920c7ae admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:06.440358 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06083932-dfe8-4804-997d-bd89c920c7ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:06.444202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06083932-dfe8-4804-997d-bd89c920c7ae admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:06.476291 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1371/2738] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:35:06 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:06.483715 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06083932-dfe8-4804-997d-bd89c920c7ae admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:06.495680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06083932-dfe8-4804-997d-bd89c920c7ae admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:06.501734 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce236a-9de0-4687-901e-0e3cbea8d96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:06.502308 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce236a-9de0-4687-901e-0e3cbea8d96f admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:06.502838 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce236a-9de0-4687-901e-0e3cbea8d96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:06.503337 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce236a-9de0-4687-901e-0e3cbea8d96f admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:06.504459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce236a-9de0-4687-901e-0e3cbea8d96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:06.506301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dce236a-9de0-4687-901e-0e3cbea8d96f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:06.519094 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb681f5-780b-4644-a24c-58927ee6f329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:06.521032 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb681f5-780b-4644-a24c-58927ee6f329 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:06.530204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1372/2739] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:35:06.534846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1368/2740] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:35:06.537033 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce236a-9de0-4687-901e-0e3cbea8d96f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:06.548361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce236a-9de0-4687-901e-0e3cbea8d96f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:06.554515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e4839-4676-4373-84cf-9437fae65d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:06.554712 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4839-4676-4373-84cf-9437fae65d1e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:06.554978 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4839-4676-4373-84cf-9437fae65d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:06.555259 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e4839-4676-4373-84cf-9437fae65d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:06.556240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e4839-4676-4373-84cf-9437fae65d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:06.557601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e4839-4676-4373-84cf-9437fae65d1e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:06.584934 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1373/2741] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:06.586612 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb681f5-780b-4644-a24c-58927ee6f329 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:06.598129 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb681f5-780b-4644-a24c-58927ee6f329 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:06.603971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c630fe-7287-4fea-9b78-33e94c0413e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:06.604938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c630fe-7287-4fea-9b78-33e94c0413e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:06.605228 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c630fe-7287-4fea-9b78-33e94c0413e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:06.605509 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c630fe-7287-4fea-9b78-33e94c0413e8 admin admin] PATH_INFO: `/v3/projects/afa96968017249f28bbca7cbb5980dd3/users/e9c685f0af514b95bdbf3041ebcbf5b4/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:06.650897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c630fe-7287-4fea-9b78-33e94c0413e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa96968017249f28bbca7cbb5980dd3, user_id=e9c685f0af514b95bdbf3041ebcbf5b4, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:06.653766 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c630fe-7287-4fea-9b78-33e94c0413e8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:06.673566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1369/2742] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:06 2022] PUT /identity/v3/projects/afa96968017249f28bbca7cbb5980dd3/users/e9c685f0af514b95bdbf3041ebcbf5b4/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:06.684748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228d959-57d5-4300-b63b-83a363111769 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:06.685036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228d959-57d5-4300-b63b-83a363111769 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:06.685311 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228d959-57d5-4300-b63b-83a363111769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:06.722020 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-7228d959-57d5-4300-b63b-83a363111769 None None] MFA Rules not processed for user `e9c685f0af514b95bdbf3041ebcbf5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:06.723320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7228d959-57d5-4300-b63b-83a363111769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:06.810796 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1374/2743] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:06.824213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c630fe-7287-4fea-9b78-33e94c0413e8 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:06.825943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f9c630fe-7287-4fea-9b78-33e94c0413e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:06.896429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c630fe-7287-4fea-9b78-33e94c0413e8 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:06.901026 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d6112-07c8-4fdb-87a2-805783aa5e6f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13121a263c31433dbd4a5082b4bf3bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:06.901710 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d6112-07c8-4fdb-87a2-805783aa5e6f service glance] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:06.901994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d6112-07c8-4fdb-87a2-805783aa5e6f service glance] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:06.902319 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d6112-07c8-4fdb-87a2-805783aa5e6f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:06.912356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d6112-07c8-4fdb-87a2-805783aa5e6f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:06.913935 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d6112-07c8-4fdb-87a2-805783aa5e6f service glance] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:06.924121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1370/2744] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:07.033538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228d959-57d5-4300-b63b-83a363111769 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:07.062775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:07.107175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228d959-57d5-4300-b63b-83a363111769 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:07.112973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:07.113847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:07.114131 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:07.114408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] PATH_INFO: `/v3/users/5b1bbfbd59c749f68955e76c1d462fbb` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:07.115639 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b1bbfbd59c749f68955e76c1d462fbb)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:07.117609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:07.166157 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b1bbfbd59c749f68955e76c1d462fbb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:07.182617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b1bbfbd59c749f68955e76c1d462fbb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:07.183342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b1bbfbd59c749f68955e76c1d462fbb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:07.190691 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b1bbfbd59c749f68955e76c1d462fbb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:07.191422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b1bbfbd59c749f68955e76c1d462fbb'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:07.205564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1375/2745] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:07 2022] DELETE /identity/v3/users/5b1bbfbd59c749f68955e76c1d462fbb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:07.209957 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c002c-2ede-4177-a684-8088ca390ce7 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:07.210251 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c002c-2ede-4177-a684-8088ca390ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:07.210547 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c002c-2ede-4177-a684-8088ca390ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:07.230283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-240c002c-2ede-4177-a684-8088ca390ce7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:07.231592 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-240c002c-2ede-4177-a684-8088ca390ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:07.242057 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:07.307930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1371/2746] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:07.318666 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:07.320664 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:07.406479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaa1dc-178e-42f1-812b-212cddf8e68b admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:07.410357 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd1114-5991-49b5-a7e7-bb6fead86807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:07.411034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd1114-5991-49b5-a7e7-bb6fead86807 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:07.411319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd1114-5991-49b5-a7e7-bb6fead86807 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:07.411597 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd1114-5991-49b5-a7e7-bb6fead86807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:07.423905 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd1114-5991-49b5-a7e7-bb6fead86807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:07.425375 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd1114-5991-49b5-a7e7-bb6fead86807 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:07.436735 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1376/2747] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:07.823507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c002c-2ede-4177-a684-8088ca390ce7 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:07.825785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-240c002c-2ede-4177-a684-8088ca390ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:07.861467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c002c-2ede-4177-a684-8088ca390ce7 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:07.875934 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:07.884484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:07.884805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:07.885142 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] PATH_INFO: `/v3/projects/01e91a601fef4fe784b5e5e699b1d879` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:07.890571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01e91a601fef4fe784b5e5e699b1d879)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:07.892857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:07.966247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01e91a601fef4fe784b5e5e699b1d879', 'request_id': 'req-eb70f9c6-9114-416e-95b2-7d02f7a8f866'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:07.973191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01e91a601fef4fe784b5e5e699b1d879', 'request_id': 'req-eb70f9c6-9114-416e-95b2-7d02f7a8f866'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:07.985930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01e91a601fef4fe784b5e5e699b1d879', 'request_id': 'req-eb70f9c6-9114-416e-95b2-7d02f7a8f866'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.000969 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] The token cache is being invalidate because project 01e91a601fef4fe784b5e5e699b1d879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:08.001295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.002005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.003725 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1372/2748] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:07 2022] DELETE /identity/v3/projects/01e91a601fef4fe784b5e5e699b1d879 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:08.007808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd1114-5991-49b5-a7e7-bb6fead86807 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:08.019224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd1114-5991-49b5-a7e7-bb6fead86807 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:08.069384 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:08.088612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc6ce2-bfdc-4a85-aec4-02e0f5c139e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:08.089525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc6ce2-bfdc-4a85-aec4-02e0f5c139e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:08.089820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc6ce2-bfdc-4a85-aec4-02e0f5c139e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:08.090099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc6ce2-bfdc-4a85-aec4-02e0f5c139e1 admin admin] PATH_INFO: `/v3/users/553d913e82af452b86a0ba8a83ea4dee` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:08.091396 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc6ce2-bfdc-4a85-aec4-02e0f5c139e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=553d913e82af452b86a0ba8a83ea4dee)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:08.093522 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc6ce2-bfdc-4a85-aec4-02e0f5c139e1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:08.145828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0ebc6ce2-bfdc-4a85-aec4-02e0f5c139e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '553d913e82af452b86a0ba8a83ea4dee'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.159410 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0ebc6ce2-bfdc-4a85-aec4-02e0f5c139e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '553d913e82af452b86a0ba8a83ea4dee'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.160105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0ebc6ce2-bfdc-4a85-aec4-02e0f5c139e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '553d913e82af452b86a0ba8a83ea4dee'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.167063 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0ebc6ce2-bfdc-4a85-aec4-02e0f5c139e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '553d913e82af452b86a0ba8a83ea4dee'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.167711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0ebc6ce2-bfdc-4a85-aec4-02e0f5c139e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '553d913e82af452b86a0ba8a83ea4dee'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.184211 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1377/2749] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:08 2022] DELETE /identity/v3/users/553d913e82af452b86a0ba8a83ea4dee => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:08.503284 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:08.504919 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:08.519598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70f9c6-9114-416e-95b2-7d02f7a8f866 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:08.592907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:08.593778 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:08.594059 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:08.594334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] PATH_INFO: `/v3/projects/2aff9f1bec0f474b83ecaa647d093877` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:08.595596 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aff9f1bec0f474b83ecaa647d093877)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:08.597548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:08.645185 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:08.652903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aff9f1bec0f474b83ecaa647d093877', 'request_id': 'req-18aa8e2d-5e67-4b78-8611-8e1fb275241f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.661928 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aff9f1bec0f474b83ecaa647d093877', 'request_id': 'req-18aa8e2d-5e67-4b78-8611-8e1fb275241f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.673208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aff9f1bec0f474b83ecaa647d093877', 'request_id': 'req-18aa8e2d-5e67-4b78-8611-8e1fb275241f'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.685907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] The token cache is being invalidate because project 2aff9f1bec0f474b83ecaa647d093877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:08.686241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.686902 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-18aa8e2d-5e67-4b78-8611-8e1fb275241f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:08.688447 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1373/2750] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:08 2022] DELETE /identity/v3/projects/2aff9f1bec0f474b83ecaa647d093877 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:35:08.693767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8cfa5-d86f-4d0c-a684-32d0095a5418 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:08.694261 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8cfa5-d86f-4d0c-a684-32d0095a5418 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:08.694558 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8cfa5-d86f-4d0c-a684-32d0095a5418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:08.715246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-8bb8cfa5-d86f-4d0c-a684-32d0095a5418 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:08.716461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8bb8cfa5-d86f-4d0c-a684-32d0095a5418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:08.786824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1378/2751] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:08.794654 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f445c-15a8-403b-822e-d92c58199f47 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:08.794945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f445c-15a8-403b-822e-d92c58199f47 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:08.795223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f445c-15a8-403b-822e-d92c58199f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:08.814842 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-bc6f445c-15a8-403b-822e-d92c58199f47 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:08.816054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-bc6f445c-15a8-403b-822e-d92c58199f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:08.824239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1374/2752] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:08.828519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8cfa5-d86f-4d0c-a684-32d0095a5418 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:08.838197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8cfa5-d86f-4d0c-a684-32d0095a5418 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:08.842837 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286176e-62db-491f-806e-b86ff005d2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:08.843504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286176e-62db-491f-806e-b86ff005d2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:08.843789 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286176e-62db-491f-806e-b86ff005d2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:08.844065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2286176e-62db-491f-806e-b86ff005d2e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:08.844790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286176e-62db-491f-806e-b86ff005d2e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:35:08.845098 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286176e-62db-491f-806e-b86ff005d2e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:08.846731 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2286176e-62db-491f-806e-b86ff005d2e5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:08.859696 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1379/2753] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:35:08.863988 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f445c-15a8-403b-822e-d92c58199f47 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:08.873119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f445c-15a8-403b-822e-d92c58199f47 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:08.878455 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29b110-40c7-441f-bdce-35188e74ffc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:08.879548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29b110-40c7-441f-bdce-35188e74ffc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:08.879548 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29b110-40c7-441f-bdce-35188e74ffc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:08.879781 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29b110-40c7-441f-bdce-35188e74ffc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:08.880375 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29b110-40c7-441f-bdce-35188e74ffc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:08.882271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29b110-40c7-441f-bdce-35188e74ffc9 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:08.924070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1375/2754] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:35:08 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:08.928239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286176e-62db-491f-806e-b86ff005d2e5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:08.938804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2286176e-62db-491f-806e-b86ff005d2e5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:08.984687 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:08.995775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4632b-7ceb-497e-b019-1f35ef9ed024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:08.996594 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4632b-7ceb-497e-b019-1f35ef9ed024 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:08.996872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4632b-7ceb-497e-b019-1f35ef9ed024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:08.997146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d4632b-7ceb-497e-b019-1f35ef9ed024 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:08.998164 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4632b-7ceb-497e-b019-1f35ef9ed024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:09.000938 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d4632b-7ceb-497e-b019-1f35ef9ed024 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:09.021650 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1380/2755] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:35:08 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:35:09.025792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29b110-40c7-441f-bdce-35188e74ffc9 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:09.036730 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29b110-40c7-441f-bdce-35188e74ffc9 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:09.041452 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8309ecf-7a8e-474c-aaa4-e300f681fc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:09.042108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8309ecf-7a8e-474c-aaa4-e300f681fc77 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:09.042393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8309ecf-7a8e-474c-aaa4-e300f681fc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:09.042706 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8309ecf-7a8e-474c-aaa4-e300f681fc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:09.043436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8309ecf-7a8e-474c-aaa4-e300f681fc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:09.045054 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8309ecf-7a8e-474c-aaa4-e300f681fc77 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:09.056131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1376/2756] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:09.062629 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4632b-7ceb-497e-b019-1f35ef9ed024 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:09.075359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d4632b-7ceb-497e-b019-1f35ef9ed024 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:09.083688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3a06d7-98c8-48fc-9b10-969860c77ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:09.084427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a06d7-98c8-48fc-9b10-969860c77ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:09.084916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a06d7-98c8-48fc-9b10-969860c77ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:09.085331 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3a06d7-98c8-48fc-9b10-969860c77ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:09.086192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a06d7-98c8-48fc-9b10-969860c77ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:09.088158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3a06d7-98c8-48fc-9b10-969860c77ae1 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:09.108011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1381/2757] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:09.113131 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8309ecf-7a8e-474c-aaa4-e300f681fc77 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:09.127091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8309ecf-7a8e-474c-aaa4-e300f681fc77 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:09.133124 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e7b3f-af09-436b-897f-48a16ec65034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:09.134024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e7b3f-af09-436b-897f-48a16ec65034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:09.134306 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e7b3f-af09-436b-897f-48a16ec65034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:09.134614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e7b3f-af09-436b-897f-48a16ec65034 admin admin] PATH_INFO: `/v3/projects/6d7e589fb9104d04b59efe817ad14ac5/users/73f889ae98ef4f34bc1ebc907dc52e79/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:09.153257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e7b3f-af09-436b-897f-48a16ec65034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d7e589fb9104d04b59efe817ad14ac5, user_id=73f889ae98ef4f34bc1ebc907dc52e79, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:09.156577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1e7b3f-af09-436b-897f-48a16ec65034 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:09.171694 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1377/2758] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:09 2022] PUT /identity/v3/projects/6d7e589fb9104d04b59efe817ad14ac5/users/73f889ae98ef4f34bc1ebc907dc52e79/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:09.186304 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9c7f2-53cd-4577-8b0b-7fe31e11be88 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:09.187619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9c7f2-53cd-4577-8b0b-7fe31e11be88 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:09.187619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9c7f2-53cd-4577-8b0b-7fe31e11be88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:09.244871 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-89d9c7f2-53cd-4577-8b0b-7fe31e11be88 None None] MFA Rules not processed for user `73f889ae98ef4f34bc1ebc907dc52e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:09.246202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-89d9c7f2-53cd-4577-8b0b-7fe31e11be88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:09.335874 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1382/2759] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:09.341048 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e7b3f-af09-436b-897f-48a16ec65034 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:09.436068 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e7b3f-af09-436b-897f-48a16ec65034 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:09.442042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d86709-6887-4a08-9a70-415ab847810c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:09.442828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d86709-6887-4a08-9a70-415ab847810c admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:09.443109 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d86709-6887-4a08-9a70-415ab847810c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:09.443381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d86709-6887-4a08-9a70-415ab847810c admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:09.444084 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d86709-6887-4a08-9a70-415ab847810c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:09.446042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d86709-6887-4a08-9a70-415ab847810c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:09.506361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1378/2760] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:35:09 2022] POST /identity/v3/projects => generated 365 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:35:09.510641 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9c7f2-53cd-4577-8b0b-7fe31e11be88 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:09.527399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9c7f2-53cd-4577-8b0b-7fe31e11be88 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:09.638574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe63306-5f1b-4cd5-9d63-f18270282abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:09.640111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe63306-5f1b-4cd5-9d63-f18270282abd admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:09.640400 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe63306-5f1b-4cd5-9d63-f18270282abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:09.640680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe63306-5f1b-4cd5-9d63-f18270282abd admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:09.641378 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe63306-5f1b-4cd5-9d63-f18270282abd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:09.643549 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe63306-5f1b-4cd5-9d63-f18270282abd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:09.689581 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1383/2761] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:35:09 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:09.697651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d86709-6887-4a08-9a70-415ab847810c admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:09.715563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d86709-6887-4a08-9a70-415ab847810c admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:09.722927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf55a60-bbca-43f4-8314-0f382b89027b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:09.723712 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf55a60-bbca-43f4-8314-0f382b89027b admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:09.723903 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf55a60-bbca-43f4-8314-0f382b89027b admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:09.724177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf55a60-bbca-43f4-8314-0f382b89027b admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:09.724911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf55a60-bbca-43f4-8314-0f382b89027b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:09.727015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf55a60-bbca-43f4-8314-0f382b89027b admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:09.742587 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1379/2762] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:09.745843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe63306-5f1b-4cd5-9d63-f18270282abd admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:09.757401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe63306-5f1b-4cd5-9d63-f18270282abd admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:09.762654 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f7206-ffd3-4055-b74f-0b7558f7de36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:09.764678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f7206-ffd3-4055-b74f-0b7558f7de36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:09.765201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f7206-ffd3-4055-b74f-0b7558f7de36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:09.765492 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f7206-ffd3-4055-b74f-0b7558f7de36 admin admin] PATH_INFO: `/v3/projects/90099dce73324f1f9fcff62c8c727866/users/ba5b814928024ad7970330ff266d59cc/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:09.784446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f7206-ffd3-4055-b74f-0b7558f7de36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90099dce73324f1f9fcff62c8c727866, user_id=ba5b814928024ad7970330ff266d59cc, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:09.787392 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f7206-ffd3-4055-b74f-0b7558f7de36 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:09.801217 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1384/2763] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:09 2022] PUT /identity/v3/projects/90099dce73324f1f9fcff62c8c727866/users/ba5b814928024ad7970330ff266d59cc/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:09.804939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf55a60-bbca-43f4-8314-0f382b89027b admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:09.815757 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf55a60-bbca-43f4-8314-0f382b89027b admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:09.855423 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:09.882747 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5eb191-c583-4af1-8cd4-85fe87f0a3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:09.883428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eb191-c583-4af1-8cd4-85fe87f0a3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:09.883713 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eb191-c583-4af1-8cd4-85fe87f0a3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:09.883990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5eb191-c583-4af1-8cd4-85fe87f0a3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:09.884789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5eb191-c583-4af1-8cd4-85fe87f0a3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:09.886699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5eb191-c583-4af1-8cd4-85fe87f0a3c1 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:09.900828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1380/2764] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:09.902149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f7206-ffd3-4055-b74f-0b7558f7de36 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:09.912963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f7206-ffd3-4055-b74f-0b7558f7de36 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:09.918298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd71d67-24ff-48e4-b012-83b5905fb6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:09.919198 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd71d67-24ff-48e4-b012-83b5905fb6b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:09.919478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd71d67-24ff-48e4-b012-83b5905fb6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:09.919757 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd71d67-24ff-48e4-b012-83b5905fb6b0 admin admin] PATH_INFO: `/v3/projects/90099dce73324f1f9fcff62c8c727866/users/ba5b814928024ad7970330ff266d59cc/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:09.921919 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd71d67-24ff-48e4-b012-83b5905fb6b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90099dce73324f1f9fcff62c8c727866, user_id=ba5b814928024ad7970330ff266d59cc, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:09.925988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd71d67-24ff-48e4-b012-83b5905fb6b0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:09.940113 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1385/2765] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:09 2022] PUT /identity/v3/projects/90099dce73324f1f9fcff62c8c727866/users/ba5b814928024ad7970330ff266d59cc/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:09.951409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87256e51-6b64-4b84-8e7e-7efd164f0b8c None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:09.952691 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87256e51-6b64-4b84-8e7e-7efd164f0b8c None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:09.953085 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87256e51-6b64-4b84-8e7e-7efd164f0b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:09.989152 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-87256e51-6b64-4b84-8e7e-7efd164f0b8c None None] MFA Rules not processed for user `ba5b814928024ad7970330ff266d59cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:09.990378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-87256e51-6b64-4b84-8e7e-7efd164f0b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:10.075799 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1381/2766] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:09 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:10.088827 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd71d67-24ff-48e4-b012-83b5905fb6b0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:10.091035 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-dcd71d67-24ff-48e4-b012-83b5905fb6b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:10.112471 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:10.163531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd71d67-24ff-48e4-b012-83b5905fb6b0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:10.167245 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6423236-d262-4b54-b32f-a295ec062c0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:10.167897 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423236-d262-4b54-b32f-a295ec062c0e service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:10.168183 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423236-d262-4b54-b32f-a295ec062c0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:10.168463 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6423236-d262-4b54-b32f-a295ec062c0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:10.178865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6423236-d262-4b54-b32f-a295ec062c0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:10.181002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6423236-d262-4b54-b32f-a295ec062c0e service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:10.194426 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1386/2767] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:10 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:11.807674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87256e51-6b64-4b84-8e7e-7efd164f0b8c None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:11.876967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87256e51-6b64-4b84-8e7e-7efd164f0b8c None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:11.881817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:11.882803 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:11.883090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:11.883378 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] PATH_INFO: `/v3/users/73f889ae98ef4f34bc1ebc907dc52e79` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:11.884591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f889ae98ef4f34bc1ebc907dc52e79)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:11.886832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:11.954546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f889ae98ef4f34bc1ebc907dc52e79'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:11.965506 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:11.968624 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f889ae98ef4f34bc1ebc907dc52e79'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:11.969422 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f889ae98ef4f34bc1ebc907dc52e79'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:11.977340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f889ae98ef4f34bc1ebc907dc52e79'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:11.978017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f889ae98ef4f34bc1ebc907dc52e79'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:11.991017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1382/2768] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:11 2022] DELETE /identity/v3/users/73f889ae98ef4f34bc1ebc907dc52e79 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:11.995730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcacee-7705-4091-995f-c3c192854144 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:11.996246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcacee-7705-4091-995f-c3c192854144 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:11.996356 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcacee-7705-4091-995f-c3c192854144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:12.015482 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c1dcacee-7705-4091-995f-c3c192854144 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:12.016752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c1dcacee-7705-4091-995f-c3c192854144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:12.089990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1387/2769] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:35:12.100947 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:12.102871 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:12.166874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c68198-a5ab-4c79-b518-43ea95665158 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:12.170568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c6659-3849-4074-b0b8-066cb9d8a207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:12.171236 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c6659-3849-4074-b0b8-066cb9d8a207 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:12.171521 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c6659-3849-4074-b0b8-066cb9d8a207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:12.171797 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c6659-3849-4074-b0b8-066cb9d8a207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:12.183526 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c6659-3849-4074-b0b8-066cb9d8a207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:12.185265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c6659-3849-4074-b0b8-066cb9d8a207 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:12.197008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1383/2770] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:12.629338 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcacee-7705-4091-995f-c3c192854144 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:12.630881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c1dcacee-7705-4091-995f-c3c192854144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:12.648485 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcacee-7705-4091-995f-c3c192854144 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:12.662479 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:12.663942 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:12.666924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:12.666924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] PATH_INFO: `/v3/projects/6d7e589fb9104d04b59efe817ad14ac5` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:12.666924 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d7e589fb9104d04b59efe817ad14ac5)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:12.674305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:12.741744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d7e589fb9104d04b59efe817ad14ac5', 'request_id': 'req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:12.756317 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d7e589fb9104d04b59efe817ad14ac5', 'request_id': 'req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:12.768056 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d7e589fb9104d04b59efe817ad14ac5', 'request_id': 'req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:12.788012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] The token cache is being invalidate because project 6d7e589fb9104d04b59efe817ad14ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:12.788332 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:12.789324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:12.791283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1388/2771] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:12 2022] DELETE /identity/v3/projects/6d7e589fb9104d04b59efe817ad14ac5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:12.798121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c6659-3849-4074-b0b8-066cb9d8a207 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:12.815260 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c6659-3849-4074-b0b8-066cb9d8a207 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:12.896055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729fbcc-84c3-4ca2-8c09-7ced244f0aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:12.897693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729fbcc-84c3-4ca2-8c09-7ced244f0aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:12.898790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729fbcc-84c3-4ca2-8c09-7ced244f0aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:12.899945 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729fbcc-84c3-4ca2-8c09-7ced244f0aa4 admin admin] PATH_INFO: `/v3/users/ba5b814928024ad7970330ff266d59cc` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:12.901861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729fbcc-84c3-4ca2-8c09-7ced244f0aa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba5b814928024ad7970330ff266d59cc)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:12.906061 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729fbcc-84c3-4ca2-8c09-7ced244f0aa4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:12.981679 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-9729fbcc-84c3-4ca2-8c09-7ced244f0aa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba5b814928024ad7970330ff266d59cc'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:12.995634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-9729fbcc-84c3-4ca2-8c09-7ced244f0aa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba5b814928024ad7970330ff266d59cc'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:12.996356 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-9729fbcc-84c3-4ca2-8c09-7ced244f0aa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba5b814928024ad7970330ff266d59cc'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:13.003170 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-9729fbcc-84c3-4ca2-8c09-7ced244f0aa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba5b814928024ad7970330ff266d59cc'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:13.003801 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-9729fbcc-84c3-4ca2-8c09-7ced244f0aa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba5b814928024ad7970330ff266d59cc'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:13.013752 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:13.018840 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1384/2772] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:12 2022] DELETE /identity/v3/users/ba5b814928024ad7970330ff266d59cc => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:13.364506 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:13.366112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:13.380039 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae8625-e324-4b27-8960-83f5bd8c1eb9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:13.436147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:13.464991 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:13.465881 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:13.466154 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:13.466432 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] PATH_INFO: `/v3/projects/90099dce73324f1f9fcff62c8c727866` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:13.468284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90099dce73324f1f9fcff62c8c727866)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:13.470324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:13.528291 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90099dce73324f1f9fcff62c8c727866', 'request_id': 'req-39ce891e-73c1-4950-852f-93e4352f7771'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:13.534977 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90099dce73324f1f9fcff62c8c727866', 'request_id': 'req-39ce891e-73c1-4950-852f-93e4352f7771'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:13.543406 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90099dce73324f1f9fcff62c8c727866', 'request_id': 'req-39ce891e-73c1-4950-852f-93e4352f7771'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:13.554795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] The token cache is being invalidate because project 90099dce73324f1f9fcff62c8c727866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:13.555078 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:13.555717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-39ce891e-73c1-4950-852f-93e4352f7771 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:13.557248 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1389/2773] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:13 2022] DELETE /identity/v3/projects/90099dce73324f1f9fcff62c8c727866 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:13.562733 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60575aca-5a30-4144-926f-e71d794e9bf8 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:13.563029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60575aca-5a30-4144-926f-e71d794e9bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:13.563311 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60575aca-5a30-4144-926f-e71d794e9bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:13.582656 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-60575aca-5a30-4144-926f-e71d794e9bf8 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:13.583880 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-60575aca-5a30-4144-926f-e71d794e9bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:13.651684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1385/2774] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:13.660021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8558902-d8cc-411e-8bf7-a74f7b8632e7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:13.660431 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8558902-d8cc-411e-8bf7-a74f7b8632e7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:13.660748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8558902-d8cc-411e-8bf7-a74f7b8632e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:13.680383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-f8558902-d8cc-411e-8bf7-a74f7b8632e7 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:13.681583 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f8558902-d8cc-411e-8bf7-a74f7b8632e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:13.689930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1390/2775] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:13.694128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60575aca-5a30-4144-926f-e71d794e9bf8 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:13.704673 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60575aca-5a30-4144-926f-e71d794e9bf8 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:13.709409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba59374-6bd7-4bee-b0c9-1f92fa1ec960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:13.710063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba59374-6bd7-4bee-b0c9-1f92fa1ec960 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:13.710376 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba59374-6bd7-4bee-b0c9-1f92fa1ec960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:13.710684 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba59374-6bd7-4bee-b0c9-1f92fa1ec960 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:13.711477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba59374-6bd7-4bee-b0c9-1f92fa1ec960 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:35:13.711784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba59374-6bd7-4bee-b0c9-1f92fa1ec960 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:13.713432 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba59374-6bd7-4bee-b0c9-1f92fa1ec960 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:13.725112 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1386/2776] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:35:13.730324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8558902-d8cc-411e-8bf7-a74f7b8632e7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:13.741689 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8558902-d8cc-411e-8bf7-a74f7b8632e7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:13.746451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b15e07-5e24-4122-8ebc-6e7581faf9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:13.747138 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b15e07-5e24-4122-8ebc-6e7581faf9a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:13.747422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b15e07-5e24-4122-8ebc-6e7581faf9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:13.747694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b15e07-5e24-4122-8ebc-6e7581faf9a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:13.748385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b15e07-5e24-4122-8ebc-6e7581faf9a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:13.750848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b15e07-5e24-4122-8ebc-6e7581faf9a5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:13.790840 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1391/2777] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:35:13 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:13.796537 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba59374-6bd7-4bee-b0c9-1f92fa1ec960 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:13.811886 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba59374-6bd7-4bee-b0c9-1f92fa1ec960 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:13.870440 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48ac48-bd1c-4149-a5f2-a0d572c39655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:13.871216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48ac48-bd1c-4149-a5f2-a0d572c39655 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:13.871503 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48ac48-bd1c-4149-a5f2-a0d572c39655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:13.871811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48ac48-bd1c-4149-a5f2-a0d572c39655 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:13.872510 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48ac48-bd1c-4149-a5f2-a0d572c39655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:13.875019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48ac48-bd1c-4149-a5f2-a0d572c39655 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:13.896060 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1387/2778] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:35:13 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:13.901055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b15e07-5e24-4122-8ebc-6e7581faf9a5 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:13.911427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b15e07-5e24-4122-8ebc-6e7581faf9a5 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:13.916147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29d455-1269-4c58-acfb-53698f7ae0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:13.916804 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29d455-1269-4c58-acfb-53698f7ae0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:13.917250 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29d455-1269-4c58-acfb-53698f7ae0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:13.917517 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29d455-1269-4c58-acfb-53698f7ae0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:13.918242 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29d455-1269-4c58-acfb-53698f7ae0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:13.919879 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29d455-1269-4c58-acfb-53698f7ae0c2 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:13.931061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1392/2779] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:13.935164 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48ac48-bd1c-4149-a5f2-a0d572c39655 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:13.949259 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48ac48-bd1c-4149-a5f2-a0d572c39655 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:13.954127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075b406-f2d4-4178-8f8b-33fec44fd93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:13.954950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075b406-f2d4-4178-8f8b-33fec44fd93c admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:13.955234 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075b406-f2d4-4178-8f8b-33fec44fd93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:13.955772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075b406-f2d4-4178-8f8b-33fec44fd93c admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:13.956804 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075b406-f2d4-4178-8f8b-33fec44fd93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:13.958792 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075b406-f2d4-4178-8f8b-33fec44fd93c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:13.970453 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1388/2780] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:13.974838 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29d455-1269-4c58-acfb-53698f7ae0c2 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:13.985886 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29d455-1269-4c58-acfb-53698f7ae0c2 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:13.990669 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90846490-5c32-498d-a5c5-66a3af9adfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:13.991560 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90846490-5c32-498d-a5c5-66a3af9adfef admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:13.991845 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90846490-5c32-498d-a5c5-66a3af9adfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:13.992122 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90846490-5c32-498d-a5c5-66a3af9adfef admin admin] PATH_INFO: `/v3/projects/13dbfa3c32694922b872d59dce06c422/users/12e4f38e3f394b4ca44463231fab92f8/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:14.009228 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90846490-5c32-498d-a5c5-66a3af9adfef admin admin] RBAC: Authorizing `identity:create_grant(project_id=13dbfa3c32694922b872d59dce06c422, user_id=12e4f38e3f394b4ca44463231fab92f8, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:14.012199 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90846490-5c32-498d-a5c5-66a3af9adfef admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:14.034267 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1393/2781] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:13 2022] PUT /identity/v3/projects/13dbfa3c32694922b872d59dce06c422/users/12e4f38e3f394b4ca44463231fab92f8/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:14.039689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052af8b-4486-4772-8f5b-d14df98b0af4 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:14.040204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052af8b-4486-4772-8f5b-d14df98b0af4 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:14.040830 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052af8b-4486-4772-8f5b-d14df98b0af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:14.074662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-7052af8b-4486-4772-8f5b-d14df98b0af4 None None] MFA Rules not processed for user `12e4f38e3f394b4ca44463231fab92f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:14.075885 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7052af8b-4486-4772-8f5b-d14df98b0af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:14.159496 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:14.175301 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1389/2782] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:35:14.176792 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90846490-5c32-498d-a5c5-66a3af9adfef admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:14.265239 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90846490-5c32-498d-a5c5-66a3af9adfef admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:14.270320 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab374248-a1c2-4582-a13d-20bf4659bb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:14.271008 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab374248-a1c2-4582-a13d-20bf4659bb39 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:14.271326 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab374248-a1c2-4582-a13d-20bf4659bb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:14.271592 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab374248-a1c2-4582-a13d-20bf4659bb39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:14.272318 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab374248-a1c2-4582-a13d-20bf4659bb39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:14.274219 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab374248-a1c2-4582-a13d-20bf4659bb39 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:14.285478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:14.304154 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1394/2783] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:35:14 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 03 15:35:14.308306 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052af8b-4486-4772-8f5b-d14df98b0af4 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:14.319617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052af8b-4486-4772-8f5b-d14df98b0af4 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:14.375696 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b58c86-ecd9-4d77-8b8d-d5237ffc2c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:14.376364 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b58c86-ecd9-4d77-8b8d-d5237ffc2c77 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:14.376648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b58c86-ecd9-4d77-8b8d-d5237ffc2c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:14.376925 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b58c86-ecd9-4d77-8b8d-d5237ffc2c77 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:14.377607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b58c86-ecd9-4d77-8b8d-d5237ffc2c77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:14.379605 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b58c86-ecd9-4d77-8b8d-d5237ffc2c77 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:14.400329 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1390/2784] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:35:14 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:14.404361 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab374248-a1c2-4582-a13d-20bf4659bb39 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:14.416590 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab374248-a1c2-4582-a13d-20bf4659bb39 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:14.421453 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e819a43-e2aa-416d-a1c5-5e2c4f499be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:14.422122 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e819a43-e2aa-416d-a1c5-5e2c4f499be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:14.422405 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e819a43-e2aa-416d-a1c5-5e2c4f499be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:14.422720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e819a43-e2aa-416d-a1c5-5e2c4f499be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:14.423443 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e819a43-e2aa-416d-a1c5-5e2c4f499be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:14.425123 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e819a43-e2aa-416d-a1c5-5e2c4f499be0 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:14.436833 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1395/2785] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:14.444334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b58c86-ecd9-4d77-8b8d-d5237ffc2c77 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:14.453393 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b58c86-ecd9-4d77-8b8d-d5237ffc2c77 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:14.458157 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729e06-a4d8-463a-a89c-3ecf50e47f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:14.459056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729e06-a4d8-463a-a89c-3ecf50e47f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:14.459334 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729e06-a4d8-463a-a89c-3ecf50e47f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:14.459602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729e06-a4d8-463a-a89c-3ecf50e47f6d admin admin] PATH_INFO: `/v3/projects/d8a82e5b16a94d1abbbfbdb27f439cfe/users/a675c0eaa5b04cdfba449bb4e2b0a747/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:14.474791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc729e06-a4d8-463a-a89c-3ecf50e47f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a82e5b16a94d1abbbfbdb27f439cfe, user_id=a675c0eaa5b04cdfba449bb4e2b0a747, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:14.477467 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc729e06-a4d8-463a-a89c-3ecf50e47f6d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:14.493563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1391/2786] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:14 2022] PUT /identity/v3/projects/d8a82e5b16a94d1abbbfbdb27f439cfe/users/a675c0eaa5b04cdfba449bb4e2b0a747/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:35:14.497587 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e819a43-e2aa-416d-a1c5-5e2c4f499be0 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:14.509031 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e819a43-e2aa-416d-a1c5-5e2c4f499be0 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:14.576087 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667cb95-7e56-46a4-8176-af3df853aad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:14.577359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667cb95-7e56-46a4-8176-af3df853aad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:14.577657 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667cb95-7e56-46a4-8176-af3df853aad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:14.578127 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667cb95-7e56-46a4-8176-af3df853aad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:14.579054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667cb95-7e56-46a4-8176-af3df853aad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:14.580765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7667cb95-7e56-46a4-8176-af3df853aad5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:14.592080 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1396/2787] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:14.596635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729e06-a4d8-463a-a89c-3ecf50e47f6d admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:14.607143 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729e06-a4d8-463a-a89c-3ecf50e47f6d admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:14.611974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96220195-d855-445e-b990-0e82e234588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:14.612859 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96220195-d855-445e-b990-0e82e234588d admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:14.613160 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96220195-d855-445e-b990-0e82e234588d admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:14.613442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96220195-d855-445e-b990-0e82e234588d admin admin] PATH_INFO: `/v3/projects/d8a82e5b16a94d1abbbfbdb27f439cfe/users/a675c0eaa5b04cdfba449bb4e2b0a747/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:14.615442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96220195-d855-445e-b990-0e82e234588d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a82e5b16a94d1abbbfbdb27f439cfe, user_id=a675c0eaa5b04cdfba449bb4e2b0a747, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:14.618090 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96220195-d855-445e-b990-0e82e234588d admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:14.629615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1392/2788] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:14 2022] PUT /identity/v3/projects/d8a82e5b16a94d1abbbfbdb27f439cfe/users/a675c0eaa5b04cdfba449bb4e2b0a747/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:14.641190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf609e-37a4-48d8-b174-8b3d1348f0f7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:14.641478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf609e-37a4-48d8-b174-8b3d1348f0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:14.641783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf609e-37a4-48d8-b174-8b3d1348f0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:14.669675 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-a3cf609e-37a4-48d8-b174-8b3d1348f0f7 None None] MFA Rules not processed for user `a675c0eaa5b04cdfba449bb4e2b0a747`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:14.671469 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-a3cf609e-37a4-48d8-b174-8b3d1348f0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:14.744579 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1397/2789] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:14.756372 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96220195-d855-445e-b990-0e82e234588d admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:14.758047 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-96220195-d855-445e-b990-0e82e234588d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:14.815592 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96220195-d855-445e-b990-0e82e234588d admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:14.819272 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e93a2-311a-4df9-8c9a-eae56a871dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:14.819912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e93a2-311a-4df9-8c9a-eae56a871dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:14.820197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e93a2-311a-4df9-8c9a-eae56a871dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:14.820475 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e93a2-311a-4df9-8c9a-eae56a871dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:14.829577 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e93a2-311a-4df9-8c9a-eae56a871dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:14.831235 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906e93a2-311a-4df9-8c9a-eae56a871dc3 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:14.842884 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1393/2790] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:15.409638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf609e-37a4-48d8-b174-8b3d1348f0f7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:15.471937 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf609e-37a4-48d8-b174-8b3d1348f0f7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:15.476462 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:15.477323 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:15.477609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:15.477879 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] PATH_INFO: `/v3/users/12e4f38e3f394b4ca44463231fab92f8` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:15.479222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e4f38e3f394b4ca44463231fab92f8)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:15.481158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:15.483554 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:15.534478 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e4f38e3f394b4ca44463231fab92f8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:15.547059 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e4f38e3f394b4ca44463231fab92f8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:15.547725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e4f38e3f394b4ca44463231fab92f8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:15.554906 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e4f38e3f394b4ca44463231fab92f8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:15.555538 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e4f38e3f394b4ca44463231fab92f8'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:15.569694 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1398/2791] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:15 2022] DELETE /identity/v3/users/12e4f38e3f394b4ca44463231fab92f8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:15.574385 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96799be-32d6-4014-a84d-6655c62e14de None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:15.575191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96799be-32d6-4014-a84d-6655c62e14de None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:15.575191 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96799be-32d6-4014-a84d-6655c62e14de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:15.593857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-c96799be-32d6-4014-a84d-6655c62e14de None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:15.595089 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c96799be-32d6-4014-a84d-6655c62e14de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:15.650697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:15.666613 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1394/2792] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:35:15.678188 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:15.679864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:15.776531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e801707-6b4e-4446-b45e-f5b59cb1bb98 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:15.780913 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6625261-f35a-4b87-aea6-0e69e04124ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:15.781570 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6625261-f35a-4b87-aea6-0e69e04124ea service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:15.781852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6625261-f35a-4b87-aea6-0e69e04124ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:15.782124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6625261-f35a-4b87-aea6-0e69e04124ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:15.792106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6625261-f35a-4b87-aea6-0e69e04124ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:15.793674 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6625261-f35a-4b87-aea6-0e69e04124ea service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:15.807238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1399/2793] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:16.154036 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96799be-32d6-4014-a84d-6655c62e14de None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:16.156913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c96799be-32d6-4014-a84d-6655c62e14de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:16.170273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96799be-32d6-4014-a84d-6655c62e14de None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:16.177102 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:16.178005 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:16.178310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:16.178674 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] PATH_INFO: `/v3/projects/13dbfa3c32694922b872d59dce06c422` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:16.181617 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13dbfa3c32694922b872d59dce06c422)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:16.183716 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:16.234554 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13dbfa3c32694922b872d59dce06c422', 'request_id': 'req-e06d2f62-9760-4ea5-8b00-2ac7b4194064'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.240118 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13dbfa3c32694922b872d59dce06c422', 'request_id': 'req-e06d2f62-9760-4ea5-8b00-2ac7b4194064'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.248398 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13dbfa3c32694922b872d59dce06c422', 'request_id': 'req-e06d2f62-9760-4ea5-8b00-2ac7b4194064'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.259463 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] The token cache is being invalidate because project 13dbfa3c32694922b872d59dce06c422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:16.259753 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.260391 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.262204 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1395/2794] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:16 2022] DELETE /identity/v3/projects/13dbfa3c32694922b872d59dce06c422 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:16.267574 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6625261-f35a-4b87-aea6-0e69e04124ea service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:16.283710 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6625261-f35a-4b87-aea6-0e69e04124ea service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:16.354677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750be03-4a02-4969-a0ac-3072c4eda6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:16.355543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750be03-4a02-4969-a0ac-3072c4eda6d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:16.355825 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750be03-4a02-4969-a0ac-3072c4eda6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:16.356096 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750be03-4a02-4969-a0ac-3072c4eda6d4 admin admin] PATH_INFO: `/v3/users/a675c0eaa5b04cdfba449bb4e2b0a747` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:16.357272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750be03-4a02-4969-a0ac-3072c4eda6d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a675c0eaa5b04cdfba449bb4e2b0a747)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:16.359252 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750be03-4a02-4969-a0ac-3072c4eda6d4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:16.388210 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:16.412714 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1750be03-4a02-4969-a0ac-3072c4eda6d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a675c0eaa5b04cdfba449bb4e2b0a747'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.427001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1750be03-4a02-4969-a0ac-3072c4eda6d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a675c0eaa5b04cdfba449bb4e2b0a747'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.428507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1750be03-4a02-4969-a0ac-3072c4eda6d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a675c0eaa5b04cdfba449bb4e2b0a747'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.446301 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1750be03-4a02-4969-a0ac-3072c4eda6d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a675c0eaa5b04cdfba449bb4e2b0a747'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.448495 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1750be03-4a02-4969-a0ac-3072c4eda6d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a675c0eaa5b04cdfba449bb4e2b0a747'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.469974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1400/2795] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:16 2022] DELETE /identity/v3/users/a675c0eaa5b04cdfba449bb4e2b0a747 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:16.795917 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:16.797539 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:16.816965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d2f62-9760-4ea5-8b00-2ac7b4194064 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:16.890566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:16.890566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:16.890566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:16.890566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] PATH_INFO: `/v3/projects/d8a82e5b16a94d1abbbfbdb27f439cfe` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:16.891347 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a82e5b16a94d1abbbfbdb27f439cfe)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:16.891347 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:16.953242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a82e5b16a94d1abbbfbdb27f439cfe', 'request_id': 'req-672440af-4f70-4ad8-982a-db7f415cb021'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.967602 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a82e5b16a94d1abbbfbdb27f439cfe', 'request_id': 'req-672440af-4f70-4ad8-982a-db7f415cb021'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:16.981630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a82e5b16a94d1abbbfbdb27f439cfe', 'request_id': 'req-672440af-4f70-4ad8-982a-db7f415cb021'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:17.003424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] The token cache is being invalidate because project d8a82e5b16a94d1abbbfbdb27f439cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:17.003723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:17.004407 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-672440af-4f70-4ad8-982a-db7f415cb021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:17.009339 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1396/2796] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:16 2022] DELETE /identity/v3/projects/d8a82e5b16a94d1abbbfbdb27f439cfe => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:17.011266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb2071-f603-4d5f-8dfa-dbef6e3c6db6 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:17.011655 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb2071-f603-4d5f-8dfa-dbef6e3c6db6 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:17.011910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb2071-f603-4d5f-8dfa-dbef6e3c6db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:17.038128 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c4cb2071-f603-4d5f-8dfa-dbef6e3c6db6 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:17.039470 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c4cb2071-f603-4d5f-8dfa-dbef6e3c6db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:17.123007 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1401/2797] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:35:17.132751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb086e-0074-42c3-b812-79c757cdcb30 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:17.133031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb086e-0074-42c3-b812-79c757cdcb30 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:17.133429 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb086e-0074-42c3-b812-79c757cdcb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:17.141512 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:17.154899 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-74fb086e-0074-42c3-b812-79c757cdcb30 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:17.156430 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-74fb086e-0074-42c3-b812-79c757cdcb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:17.164857 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1397/2798] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:17.169344 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb2071-f603-4d5f-8dfa-dbef6e3c6db6 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:17.180002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb2071-f603-4d5f-8dfa-dbef6e3c6db6 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:17.185885 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5d8f4-0522-4cb5-9b8b-6998e614b88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:17.187537 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5d8f4-0522-4cb5-9b8b-6998e614b88c admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:17.187993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5d8f4-0522-4cb5-9b8b-6998e614b88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:17.188459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5d8f4-0522-4cb5-9b8b-6998e614b88c admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:17.189695 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5d8f4-0522-4cb5-9b8b-6998e614b88c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:35:17.190244 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5d8f4-0522-4cb5-9b8b-6998e614b88c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:17.192089 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd5d8f4-0522-4cb5-9b8b-6998e614b88c admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:17.206083 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1402/2799] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:35:17.210839 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb086e-0074-42c3-b812-79c757cdcb30 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:17.222413 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb086e-0074-42c3-b812-79c757cdcb30 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:17.230045 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd414d-967c-4c1f-b7fb-fb833ae054c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:17.230761 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd414d-967c-4c1f-b7fb-fb833ae054c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:17.231049 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd414d-967c-4c1f-b7fb-fb833ae054c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:17.231349 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd414d-967c-4c1f-b7fb-fb833ae054c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:17.232063 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd414d-967c-4c1f-b7fb-fb833ae054c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:17.235662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd414d-967c-4c1f-b7fb-fb833ae054c4 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:17.263985 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1398/2800] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:35:17 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:17.270823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5d8f4-0522-4cb5-9b8b-6998e614b88c admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:17.281775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5d8f4-0522-4cb5-9b8b-6998e614b88c admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:17.515529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ec465-a189-41bd-b2f8-4300d9d5f284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:17.516398 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ec465-a189-41bd-b2f8-4300d9d5f284 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:17.516798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ec465-a189-41bd-b2f8-4300d9d5f284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:17.517112 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ec465-a189-41bd-b2f8-4300d9d5f284 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:17.518095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ec465-a189-41bd-b2f8-4300d9d5f284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:17.520848 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ec465-a189-41bd-b2f8-4300d9d5f284 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:17.547048 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1403/2801] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:35:17 2022] POST /identity/v3/users => generated 293 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:17.551327 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd414d-967c-4c1f-b7fb-fb833ae054c4 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:17.561628 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd414d-967c-4c1f-b7fb-fb833ae054c4 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:17.566281 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3ee8-0891-4cfc-93fa-cf8529782b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:17.566978 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3ee8-0891-4cfc-93fa-cf8529782b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:17.567266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3ee8-0891-4cfc-93fa-cf8529782b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:17.567563 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3ee8-0891-4cfc-93fa-cf8529782b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:17.568296 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b3ee8-0891-4cfc-93fa-cf8529782b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:17.569907 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b3ee8-0891-4cfc-93fa-cf8529782b32 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:17.580831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1399/2802] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:17.584928 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ec465-a189-41bd-b2f8-4300d9d5f284 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:17.596383 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ec465-a189-41bd-b2f8-4300d9d5f284 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:17.602702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3530e1-2f07-400c-a483-80b7bce1a3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:17.603510 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3530e1-2f07-400c-a483-80b7bce1a3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:17.603903 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3530e1-2f07-400c-a483-80b7bce1a3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:17.604247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3530e1-2f07-400c-a483-80b7bce1a3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:17.606086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3530e1-2f07-400c-a483-80b7bce1a3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:17.608091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3530e1-2f07-400c-a483-80b7bce1a3a5 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:17.620459 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1404/2803] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:17.624517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3ee8-0891-4cfc-93fa-cf8529782b32 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:17.635024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3ee8-0891-4cfc-93fa-cf8529782b32 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:17.640836 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5c2f0-d459-4872-bfd4-e280b629eeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:17.641735 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5c2f0-d459-4872-bfd4-e280b629eeb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:17.642070 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5c2f0-d459-4872-bfd4-e280b629eeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:17.642347 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5c2f0-d459-4872-bfd4-e280b629eeb1 admin admin] PATH_INFO: `/v3/projects/99f44d7e574b40d094c477cc2d53ba5a/users/75ba060bc80e46db89d4e34452c05956/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:17.663940 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5c2f0-d459-4872-bfd4-e280b629eeb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f44d7e574b40d094c477cc2d53ba5a, user_id=75ba060bc80e46db89d4e34452c05956, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:17.666630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5c2f0-d459-4872-bfd4-e280b629eeb1 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:17.685860 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1400/2804] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:17 2022] PUT /identity/v3/projects/99f44d7e574b40d094c477cc2d53ba5a/users/75ba060bc80e46db89d4e34452c05956/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:35:17.699060 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5aee9-8974-43de-8fec-4d59e43e3bc7 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:17.699504 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5aee9-8974-43de-8fec-4d59e43e3bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:17.699857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5aee9-8974-43de-8fec-4d59e43e3bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:17.725401 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:17.738930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-77f5aee9-8974-43de-8fec-4d59e43e3bc7 None None] MFA Rules not processed for user `75ba060bc80e46db89d4e34452c05956`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:17.740381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-77f5aee9-8974-43de-8fec-4d59e43e3bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:17.828547 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1405/2805] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:17.833078 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5c2f0-d459-4872-bfd4-e280b629eeb1 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:17.897785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5c2f0-d459-4872-bfd4-e280b629eeb1 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:17.902612 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a851a59-dff9-41bc-9b82-8aea686b1c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:17.903310 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a851a59-dff9-41bc-9b82-8aea686b1c78 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:17.903586 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a851a59-dff9-41bc-9b82-8aea686b1c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:17.903856 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a851a59-dff9-41bc-9b82-8aea686b1c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:17.904555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a851a59-dff9-41bc-9b82-8aea686b1c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:17.907178 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a851a59-dff9-41bc-9b82-8aea686b1c78 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:17.936148 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1401/2806] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:35:17 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:17.941539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5aee9-8974-43de-8fec-4d59e43e3bc7 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:17.954339 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5aee9-8974-43de-8fec-4d59e43e3bc7 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:18.020889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5e89f-fdc8-4b1b-ae9a-123cb69db91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:18.021895 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e89f-fdc8-4b1b-ae9a-123cb69db91f admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:18.022254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e89f-fdc8-4b1b-ae9a-123cb69db91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:18.022553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5e89f-fdc8-4b1b-ae9a-123cb69db91f admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:18.023290 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5e89f-fdc8-4b1b-ae9a-123cb69db91f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:18.025224 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5e89f-fdc8-4b1b-ae9a-123cb69db91f admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:18.046711 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1406/2807] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:35:17 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:18.051490 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a851a59-dff9-41bc-9b82-8aea686b1c78 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:18.062445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a851a59-dff9-41bc-9b82-8aea686b1c78 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:18.067332 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e538b-6bd9-460f-9e92-d88bdde41a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:18.068024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e538b-6bd9-460f-9e92-d88bdde41a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:18.068308 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e538b-6bd9-460f-9e92-d88bdde41a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:18.068589 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e538b-6bd9-460f-9e92-d88bdde41a06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:18.069431 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e538b-6bd9-460f-9e92-d88bdde41a06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:18.071098 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e538b-6bd9-460f-9e92-d88bdde41a06 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:18.082610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1402/2808] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:18.086973 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5e89f-fdc8-4b1b-ae9a-123cb69db91f admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:18.097163 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5e89f-fdc8-4b1b-ae9a-123cb69db91f admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:18.105555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29630bcf-eac4-4973-9b66-7f981e6c06fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:18.105555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29630bcf-eac4-4973-9b66-7f981e6c06fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:18.105555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29630bcf-eac4-4973-9b66-7f981e6c06fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:18.105555 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29630bcf-eac4-4973-9b66-7f981e6c06fe admin admin] PATH_INFO: `/v3/projects/3cea8cd2fb1d4e1f8d41270030f0e330/users/bc2f74c7452b4b7eb5c12ff2338b5e0c/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:18.128725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29630bcf-eac4-4973-9b66-7f981e6c06fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cea8cd2fb1d4e1f8d41270030f0e330, user_id=bc2f74c7452b4b7eb5c12ff2338b5e0c, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:18.131702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29630bcf-eac4-4973-9b66-7f981e6c06fe admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:18.148788 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1407/2809] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:18 2022] PUT /identity/v3/projects/3cea8cd2fb1d4e1f8d41270030f0e330/users/bc2f74c7452b4b7eb5c12ff2338b5e0c/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:18.154208 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e538b-6bd9-460f-9e92-d88bdde41a06 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:18.171874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e538b-6bd9-460f-9e92-d88bdde41a06 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:18.249648 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1765af5e-51f6-4fe7-883b-d8c2a53d7fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:18.250697 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765af5e-51f6-4fe7-883b-d8c2a53d7fab admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:18.250981 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765af5e-51f6-4fe7-883b-d8c2a53d7fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:18.251255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1765af5e-51f6-4fe7-883b-d8c2a53d7fab admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:18.252111 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765af5e-51f6-4fe7-883b-d8c2a53d7fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:18.253798 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1765af5e-51f6-4fe7-883b-d8c2a53d7fab admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:18.265421 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1403/2810] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:18.269747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29630bcf-eac4-4973-9b66-7f981e6c06fe admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:18.280745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29630bcf-eac4-4973-9b66-7f981e6c06fe admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:18.301369 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e94f2-d7a7-4b27-b392-73b424d1fbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:18.302269 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e94f2-d7a7-4b27-b392-73b424d1fbb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:18.302553 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e94f2-d7a7-4b27-b392-73b424d1fbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:18.302862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e94f2-d7a7-4b27-b392-73b424d1fbb6 admin admin] PATH_INFO: `/v3/projects/3cea8cd2fb1d4e1f8d41270030f0e330/users/bc2f74c7452b4b7eb5c12ff2338b5e0c/roles/d67707782358417ba9f25b2793ca2890` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:18.304822 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e94f2-d7a7-4b27-b392-73b424d1fbb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cea8cd2fb1d4e1f8d41270030f0e330, user_id=bc2f74c7452b4b7eb5c12ff2338b5e0c, role_id=d67707782358417ba9f25b2793ca2890)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:18.307536 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e94f2-d7a7-4b27-b392-73b424d1fbb6 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:18.319645 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1408/2811] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:18 2022] PUT /identity/v3/projects/3cea8cd2fb1d4e1f8d41270030f0e330/users/bc2f74c7452b4b7eb5c12ff2338b5e0c/roles/d67707782358417ba9f25b2793ca2890 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:18.330858 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47e859-ecf5-49c2-8713-de6b13d4e3cf None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:18.331147 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47e859-ecf5-49c2-8713-de6b13d4e3cf None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:18.331491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47e859-ecf5-49c2-8713-de6b13d4e3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:18.352481 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:18.362083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-ce47e859-ecf5-49c2-8713-de6b13d4e3cf None None] MFA Rules not processed for user `bc2f74c7452b4b7eb5c12ff2338b5e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:18.363485 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-ce47e859-ecf5-49c2-8713-de6b13d4e3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:18.463803 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1404/2812] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:18.476752 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e94f2-d7a7-4b27-b392-73b424d1fbb6 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:18.479407 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-623e94f2-d7a7-4b27-b392-73b424d1fbb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:18.578052 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e94f2-d7a7-4b27-b392-73b424d1fbb6 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:18.582145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a87a1-ce6f-434d-a2e3-2890b7604705 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:18.582883 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a87a1-ce6f-434d-a2e3-2890b7604705 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:18.583105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a87a1-ce6f-434d-a2e3-2890b7604705 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:18.583386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a87a1-ce6f-434d-a2e3-2890b7604705 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:18.588055 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:18.650499 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a87a1-ce6f-434d-a2e3-2890b7604705 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:18.652044 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a87a1-ce6f-434d-a2e3-2890b7604705 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:18.661266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1409/2813] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:35:22.865245 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672f2d1-4762-4fd4-aa88-cc46f93a7908 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:22.865744 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672f2d1-4762-4fd4-aa88-cc46f93a7908 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:22.865864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672f2d1-4762-4fd4-aa88-cc46f93a7908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:22.883703 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-b672f2d1-4762-4fd4-aa88-cc46f93a7908 None None] MFA Rules not processed for user `4db856168c34491080b93c8ee5680c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:22.884905 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b672f2d1-4762-4fd4-aa88-cc46f93a7908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:22.947192 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1405/2814] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:23.206135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a87a1-ce6f-434d-a2e3-2890b7604705 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:23.207670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-421a87a1-ce6f-434d-a2e3-2890b7604705 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:23.266974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a87a1-ce6f-434d-a2e3-2890b7604705 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:23.270619 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b667-4523-46d5-808b-627da4b7fd49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:23.271255 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b667-4523-46d5-808b-627da4b7fd49 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:23.271539 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b667-4523-46d5-808b-627da4b7fd49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:23.271811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c247b667-4523-46d5-808b-627da4b7fd49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:23.273916 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c247b667-4523-46d5-808b-627da4b7fd49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:23.286192 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247b667-4523-46d5-808b-627da4b7fd49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:23.288541 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c247b667-4523-46d5-808b-627da4b7fd49 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:23.297074 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1410/2815] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:24.237841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672f2d1-4762-4fd4-aa88-cc46f93a7908 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:24.239328 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b672f2d1-4762-4fd4-aa88-cc46f93a7908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:24.313832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672f2d1-4762-4fd4-aa88-cc46f93a7908 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:24.318848 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b074426-25db-47fe-b6f5-0dd6e643715f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:24.319504 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b074426-25db-47fe-b6f5-0dd6e643715f service placement] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:24.319785 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b074426-25db-47fe-b6f5-0dd6e643715f service placement] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:24.320082 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b074426-25db-47fe-b6f5-0dd6e643715f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:24.321979 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0b074426-25db-47fe-b6f5-0dd6e643715f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:24.334497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b074426-25db-47fe-b6f5-0dd6e643715f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:24.336416 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b074426-25db-47fe-b6f5-0dd6e643715f service placement] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:24.348257 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1406/2816] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:25.223864 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b667-4523-46d5-808b-627da4b7fd49 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:25.239394 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c247b667-4523-46d5-808b-627da4b7fd49 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:25.244961 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20077d-cd22-47f5-8784-e7cfa04d2926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:25.245988 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20077d-cd22-47f5-8784-e7cfa04d2926 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:25.246473 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20077d-cd22-47f5-8784-e7cfa04d2926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:25.247147 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20077d-cd22-47f5-8784-e7cfa04d2926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:25.257843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20077d-cd22-47f5-8784-e7cfa04d2926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:25.259330 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20077d-cd22-47f5-8784-e7cfa04d2926 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:25.269992 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1411/2817] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:30.532181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b074426-25db-47fe-b6f5-0dd6e643715f service placement] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:30.589569 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=100 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:30.648851 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b074426-25db-47fe-b6f5-0dd6e643715f service placement] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:30.655723 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:30.659021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:30.659633 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:30.660130 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] PATH_INFO: `/v3/users/75ba060bc80e46db89d4e34452c05956` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:30.662420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ba060bc80e46db89d4e34452c05956)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:30.664374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:30.777630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ba060bc80e46db89d4e34452c05956'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:30.801809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ba060bc80e46db89d4e34452c05956'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:30.803212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ba060bc80e46db89d4e34452c05956'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:30.814751 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ba060bc80e46db89d4e34452c05956'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:30.816029 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ba060bc80e46db89d4e34452c05956'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:30.837675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1407/2818] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:30 2022] DELETE /identity/v3/users/75ba060bc80e46db89d4e34452c05956 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:30.845302 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb818ef-3ed9-4820-a924-efe85122a957 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:30.846817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb818ef-3ed9-4820-a924-efe85122a957 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:30.847702 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb818ef-3ed9-4820-a924-efe85122a957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:30.880106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ddb818ef-3ed9-4820-a924-efe85122a957 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:30.881879 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ddb818ef-3ed9-4820-a924-efe85122a957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:30.972054 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1412/2819] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:30.983436 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:30.986271 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:31.068668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f41786-c88a-4fb3-bd1c-856295ebd47a admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:31.076197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b844778-5cce-4962-9e41-f690d0f6ed95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:31.076197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b844778-5cce-4962-9e41-f690d0f6ed95 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:31.076197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b844778-5cce-4962-9e41-f690d0f6ed95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:31.076197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b844778-5cce-4962-9e41-f690d0f6ed95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:31.086159 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b844778-5cce-4962-9e41-f690d0f6ed95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:31.087832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b844778-5cce-4962-9e41-f690d0f6ed95 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:31.099255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1408/2820] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:35:31.442593 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb818ef-3ed9-4820-a924-efe85122a957 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:31.444623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ddb818ef-3ed9-4820-a924-efe85122a957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:31.462013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb818ef-3ed9-4820-a924-efe85122a957 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:31.473946 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:31.474887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:31.475173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:31.475421 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] PATH_INFO: `/v3/projects/99f44d7e574b40d094c477cc2d53ba5a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:31.479298 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99f44d7e574b40d094c477cc2d53ba5a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:31.480795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:31.532474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:31.551798 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99f44d7e574b40d094c477cc2d53ba5a', 'request_id': 'req-1be9eb06-f90c-4fb1-8572-cabf693006e9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:31.559190 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99f44d7e574b40d094c477cc2d53ba5a', 'request_id': 'req-1be9eb06-f90c-4fb1-8572-cabf693006e9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:31.568227 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99f44d7e574b40d094c477cc2d53ba5a', 'request_id': 'req-1be9eb06-f90c-4fb1-8572-cabf693006e9'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:31.581474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] The token cache is being invalidate because project 99f44d7e574b40d094c477cc2d53ba5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:31.581763 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:31.582427 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:31.584058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1413/2821] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:31 2022] DELETE /identity/v3/projects/99f44d7e574b40d094c477cc2d53ba5a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:31.588135 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b844778-5cce-4962-9e41-f690d0f6ed95 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:31.599791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b844778-5cce-4962-9e41-f690d0f6ed95 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:31.666777 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:31.677452 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd054e-faab-4583-a638-629e40263f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:31.678671 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd054e-faab-4583-a638-629e40263f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:31.678760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd054e-faab-4583-a638-629e40263f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:31.679123 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd054e-faab-4583-a638-629e40263f8c admin admin] PATH_INFO: `/v3/users/bc2f74c7452b4b7eb5c12ff2338b5e0c` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:31.681133 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd054e-faab-4583-a638-629e40263f8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2f74c7452b4b7eb5c12ff2338b5e0c)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:31.683402 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd054e-faab-4583-a638-629e40263f8c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:31.740405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-18bd054e-faab-4583-a638-629e40263f8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2f74c7452b4b7eb5c12ff2338b5e0c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:31.755571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-18bd054e-faab-4583-a638-629e40263f8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2f74c7452b4b7eb5c12ff2338b5e0c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:31.756420 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-18bd054e-faab-4583-a638-629e40263f8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2f74c7452b4b7eb5c12ff2338b5e0c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:31.765362 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-18bd054e-faab-4583-a638-629e40263f8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2f74c7452b4b7eb5c12ff2338b5e0c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:31.766287 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-18bd054e-faab-4583-a638-629e40263f8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2f74c7452b4b7eb5c12ff2338b5e0c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:31.786262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1409/2822] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:31 2022] DELETE /identity/v3/users/bc2f74c7452b4b7eb5c12ff2338b5e0c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:32.188614 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:32.191011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:32.209052 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9eb06-f90c-4fb1-8572-cabf693006e9 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:32.287158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:32.288036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:32.288343 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:32.288635 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] PATH_INFO: `/v3/projects/3cea8cd2fb1d4e1f8d41270030f0e330` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:32.289929 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cea8cd2fb1d4e1f8d41270030f0e330)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:32.291892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:32.352200 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cea8cd2fb1d4e1f8d41270030f0e330', 'request_id': 'req-83013111-f7cd-4943-9381-a47709bd4507'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:32.357732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cea8cd2fb1d4e1f8d41270030f0e330', 'request_id': 'req-83013111-f7cd-4943-9381-a47709bd4507'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:32.369938 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cea8cd2fb1d4e1f8d41270030f0e330', 'request_id': 'req-83013111-f7cd-4943-9381-a47709bd4507'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:32.382441 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] The token cache is being invalidate because project 3cea8cd2fb1d4e1f8d41270030f0e330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:32.382765 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:32.383422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-83013111-f7cd-4943-9381-a47709bd4507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:32.385020 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1414/2823] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:32 2022] DELETE /identity/v3/projects/3cea8cd2fb1d4e1f8d41270030f0e330 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:35:32.393034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b6d1e-e3f7-4d4a-a09e-22d3d3e8e4b3 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:32.393354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b6d1e-e3f7-4d4a-a09e-22d3d3e8e4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:32.393632 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b6d1e-e3f7-4d4a-a09e-22d3d3e8e4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:32.412035 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-0d7b6d1e-e3f7-4d4a-a09e-22d3d3e8e4b3 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:32.413240 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-0d7b6d1e-e3f7-4d4a-a09e-22d3d3e8e4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:32.484038 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1410/2824] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:32.553156 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0176209-13af-4022-9859-530b36f9f1e4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:32.553739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0176209-13af-4022-9859-530b36f9f1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:32.554105 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0176209-13af-4022-9859-530b36f9f1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:32.573636 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-d0176209-13af-4022-9859-530b36f9f1e4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:32.574941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d0176209-13af-4022-9859-530b36f9f1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:32.583518 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1415/2825] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:32.588151 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b6d1e-e3f7-4d4a-a09e-22d3d3e8e4b3 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:32.604828 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b6d1e-e3f7-4d4a-a09e-22d3d3e8e4b3 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:32.609728 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010763d9-e4b8-4583-bbc3-294607650684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:32.610388 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010763d9-e4b8-4583-bbc3-294607650684 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:32.610699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010763d9-e4b8-4583-bbc3-294607650684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:32.610971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010763d9-e4b8-4583-bbc3-294607650684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:32.611701 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010763d9-e4b8-4583-bbc3-294607650684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80221) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:35:32.612007 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010763d9-e4b8-4583-bbc3-294607650684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:32.613615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010763d9-e4b8-4583-bbc3-294607650684 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:32.627121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1411/2826] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 03 15:35:32.633745 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0176209-13af-4022-9859-530b36f9f1e4 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:32.646318 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0176209-13af-4022-9859-530b36f9f1e4 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:32.652238 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa03fe-208c-4b94-b3f6-d4066941e283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:32.653202 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa03fe-208c-4b94-b3f6-d4066941e283 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:32.653623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa03fe-208c-4b94-b3f6-d4066941e283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:32.654021 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa03fe-208c-4b94-b3f6-d4066941e283 admin admin] PATH_INFO: `/v3/projects` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:32.655111 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa03fe-208c-4b94-b3f6-d4066941e283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:32.657994 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa03fe-208c-4b94-b3f6-d4066941e283 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:32.690677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1416/2827] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:35:32 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:32.695846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010763d9-e4b8-4583-bbc3-294607650684 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:32.706181 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010763d9-e4b8-4583-bbc3-294607650684 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:32.768817 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1e09a-46b8-4f6b-9170-07118503fe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:32.769486 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1e09a-46b8-4f6b-9170-07118503fe11 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:32.769874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1e09a-46b8-4f6b-9170-07118503fe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:32.770896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1e09a-46b8-4f6b-9170-07118503fe11 admin admin] PATH_INFO: `/v3/users` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:32.770896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1e09a-46b8-4f6b-9170-07118503fe11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:32.772901 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1e09a-46b8-4f6b-9170-07118503fe11 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:32.780673 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:32.799687 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1412/2828] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:35:32 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:35:32.803963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa03fe-208c-4b94-b3f6-d4066941e283 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:32.816399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa03fe-208c-4b94-b3f6-d4066941e283 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:32.822755 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff377ee-2bcd-4f22-b269-4bda4426d9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:32.823767 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff377ee-2bcd-4f22-b269-4bda4426d9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:32.824222 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff377ee-2bcd-4f22-b269-4bda4426d9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:32.824805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff377ee-2bcd-4f22-b269-4bda4426d9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:32.825974 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff377ee-2bcd-4f22-b269-4bda4426d9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:32.829151 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff377ee-2bcd-4f22-b269-4bda4426d9ca admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:32.844169 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1417/2829] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:35:32.848031 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1e09a-46b8-4f6b-9170-07118503fe11 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:32.862498 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1e09a-46b8-4f6b-9170-07118503fe11 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:32.870216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bdb14-3e79-4872-a5d9-4288468ca35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:32.871794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bdb14-3e79-4872-a5d9-4288468ca35e admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:32.872299 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bdb14-3e79-4872-a5d9-4288468ca35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:32.872795 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bdb14-3e79-4872-a5d9-4288468ca35e admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:32.874361 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bdb14-3e79-4872-a5d9-4288468ca35e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:32.877405 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bdb14-3e79-4872-a5d9-4288468ca35e admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:32.898318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1413/2830] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:35:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:35:32.898508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff377ee-2bcd-4f22-b269-4bda4426d9ca admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:32.908143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff377ee-2bcd-4f22-b269-4bda4426d9ca admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:32.913277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156e4fc-085a-4b54-bbe9-798ee81851ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:32.914270 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156e4fc-085a-4b54-bbe9-798ee81851ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:32.914603 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156e4fc-085a-4b54-bbe9-798ee81851ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:32.914847 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156e4fc-085a-4b54-bbe9-798ee81851ad admin admin] PATH_INFO: `/v3/projects/acf00e4ef6f649e69980d6ec2c4b8bea/users/134977bf87a2409481cf99d98c8b2228/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:32.918524 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:32.934805 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156e4fc-085a-4b54-bbe9-798ee81851ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf00e4ef6f649e69980d6ec2c4b8bea, user_id=134977bf87a2409481cf99d98c8b2228, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:32.937783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156e4fc-085a-4b54-bbe9-798ee81851ad admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:32.956915 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1418/2831] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:35:32 2022] PUT /identity/v3/projects/acf00e4ef6f649e69980d6ec2c4b8bea/users/134977bf87a2409481cf99d98c8b2228/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:35:32.983395 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7f86e-6a52-4dec-9f4e-32186aac7535 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:32.984282 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7f86e-6a52-4dec-9f4e-32186aac7535 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:32.985108 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7f86e-6a52-4dec-9f4e-32186aac7535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:33.026989 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-47d7f86e-6a52-4dec-9f4e-32186aac7535 None None] MFA Rules not processed for user `134977bf87a2409481cf99d98c8b2228`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:33.028139 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-47d7f86e-6a52-4dec-9f4e-32186aac7535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:33.110641 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1414/2832] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:33.122162 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156e4fc-085a-4b54-bbe9-798ee81851ad admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:33.124486 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d156e4fc-085a-4b54-bbe9-798ee81851ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:33.192774 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156e4fc-085a-4b54-bbe9-798ee81851ad admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:33.197200 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14124753-c0e6-4abd-a196-fc3b12e06cb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:33.198319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14124753-c0e6-4abd-a196-fc3b12e06cb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:33.198319 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14124753-c0e6-4abd-a196-fc3b12e06cb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:33.199051 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14124753-c0e6-4abd-a196-fc3b12e06cb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:33.210552 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14124753-c0e6-4abd-a196-fc3b12e06cb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:33.212068 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14124753-c0e6-4abd-a196-fc3b12e06cb8 service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:33.224251 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1419/2833] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:34.627318 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5decf-3742-44aa-b868-198e52ca37e9 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:34.627892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5decf-3742-44aa-b868-198e52ca37e9 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:34.628022 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5decf-3742-44aa-b868-198e52ca37e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:34.651746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-59e5decf-3742-44aa-b868-198e52ca37e9 None None] MFA Rules not processed for user `134977bf87a2409481cf99d98c8b2228`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:34.653549 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-59e5decf-3742-44aa-b868-198e52ca37e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:34.661582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1415/2834] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:40.803241 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14124753-c0e6-4abd-a196-fc3b12e06cb8 service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:40.804979 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-14124753-c0e6-4abd-a196-fc3b12e06cb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:40.945609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14124753-c0e6-4abd-a196-fc3b12e06cb8 service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:40.956268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:40.958149 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:40.958421 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:40.958884 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] PATH_INFO: `/v3/users/e9c685f0af514b95bdbf3041ebcbf5b4` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:40.962001 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9c685f0af514b95bdbf3041ebcbf5b4)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:40.966143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:41.105153 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9c685f0af514b95bdbf3041ebcbf5b4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:41.149932 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9c685f0af514b95bdbf3041ebcbf5b4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:41.154662 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9c685f0af514b95bdbf3041ebcbf5b4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:41.169002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9c685f0af514b95bdbf3041ebcbf5b4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:41.170201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9c685f0af514b95bdbf3041ebcbf5b4'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:41.193430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1420/2835] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:35:40 2022] DELETE /identity/v3/users/e9c685f0af514b95bdbf3041ebcbf5b4 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:41.201409 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6168b-72aa-4af8-811a-2f14a9fd677f None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:41.203076 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6168b-72aa-4af8-811a-2f14a9fd677f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:41.203247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6168b-72aa-4af8-811a-2f14a9fd677f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:41.228904 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-5cc6168b-72aa-4af8-811a-2f14a9fd677f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:41.230223 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5cc6168b-72aa-4af8-811a-2f14a9fd677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:41.357874 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1416/2836] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:41.376189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:41.379861 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:41.401622 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:41.537780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29343ad2-6949-41e5-ad57-1d9a4f06ba47 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:41.541910 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e246ec-79c6-4399-bcbf-289b1e690301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:41.543172 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e246ec-79c6-4399-bcbf-289b1e690301 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:41.543172 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e246ec-79c6-4399-bcbf-289b1e690301 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:41.543677 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e246ec-79c6-4399-bcbf-289b1e690301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:41.557266 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e246ec-79c6-4399-bcbf-289b1e690301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:41.558984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e246ec-79c6-4399-bcbf-289b1e690301 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:41.573850 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1421/2837] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:35:41.972042 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6168b-72aa-4af8-811a-2f14a9fd677f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:41.974688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-5cc6168b-72aa-4af8-811a-2f14a9fd677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:41.990227 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6168b-72aa-4af8-811a-2f14a9fd677f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:41.995812 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:41.997195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:41.997689 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:41.998179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] PATH_INFO: `/v3/projects/afa96968017249f28bbca7cbb5980dd3` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:42.000956 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa96968017249f28bbca7cbb5980dd3)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:42.003424 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:42.066790 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa96968017249f28bbca7cbb5980dd3', 'request_id': 'req-c3d0571b-4ef5-466c-9c01-a7098f9bc660'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:42.072615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:INSERT=3,keystone:SELECT=91,keystone:DELETE=6 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:35:42.074349 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa96968017249f28bbca7cbb5980dd3', 'request_id': 'req-c3d0571b-4ef5-466c-9c01-a7098f9bc660'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:42.084507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa96968017249f28bbca7cbb5980dd3', 'request_id': 'req-c3d0571b-4ef5-466c-9c01-a7098f9bc660'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:42.098180 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] The token cache is being invalidate because project afa96968017249f28bbca7cbb5980dd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:35:42.098598 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:42.099811 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:35:42.103401 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1417/2838] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:35:41 2022] DELETE /identity/v3/projects/afa96968017249f28bbca7cbb5980dd3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:35:47.941484 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186506d8-73b3-451f-88b2-5d122351061c None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:47.941975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186506d8-73b3-451f-88b2-5d122351061c None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:47.941975 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186506d8-73b3-451f-88b2-5d122351061c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:47.960307 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-186506d8-73b3-451f-88b2-5d122351061c None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:47.961525 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-186506d8-73b3-451f-88b2-5d122351061c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:48.018729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1422/2839] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:35:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:48.068791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:48.070248 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:48.141582 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0571b-4ef5-466c-9c01-a7098f9bc660 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:48.146927 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175608d-5c18-47b5-ab8a-fff371abbe46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:48.147581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175608d-5c18-47b5-ab8a-fff371abbe46 service swift] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:48.147863 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175608d-5c18-47b5-ab8a-fff371abbe46 service swift] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:48.148262 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175608d-5c18-47b5-ab8a-fff371abbe46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:48.158172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175608d-5c18-47b5-ab8a-fff371abbe46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:48.159876 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175608d-5c18-47b5-ab8a-fff371abbe46 service swift] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:48.170242 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1418/2840] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:35:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:35:57.554925 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f30d-44e7-446f-838a-f50d8c4e5ff2 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:57.555797 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f30d-44e7-446f-838a-f50d8c4e5ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:57.556094 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79f30d-44e7-446f-838a-f50d8c4e5ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:57.575865 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ca79f30d-44e7-446f-838a-f50d8c4e5ff2 None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:57.577475 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ca79f30d-44e7-446f-838a-f50d8c4e5ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:57.584795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1423/2841] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:35:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:57.599557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175608d-5c18-47b5-ab8a-fff371abbe46 service swift] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:57.614247 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175608d-5c18-47b5-ab8a-fff371abbe46 service swift] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:57.619450 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136713e-b8ca-427b-b3d6-ec4fa7fe8488 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:57.620323 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136713e-b8ca-427b-b3d6-ec4fa7fe8488 service swift] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:57.620767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136713e-b8ca-427b-b3d6-ec4fa7fe8488 service swift] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:57.621281 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136713e-b8ca-427b-b3d6-ec4fa7fe8488 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:57.630739 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136713e-b8ca-427b-b3d6-ec4fa7fe8488 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:57.633083 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136713e-b8ca-427b-b3d6-ec4fa7fe8488 service swift] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:57.643015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1419/2842] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:35:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:35:58.307197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38616e-aa10-4cff-b952-cf788f9d48fd None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:58.307758 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38616e-aa10-4cff-b952-cf788f9d48fd None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:58.308707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38616e-aa10-4cff-b952-cf788f9d48fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:58.336678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-ef38616e-aa10-4cff-b952-cf788f9d48fd None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:58.338143 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-ef38616e-aa10-4cff-b952-cf788f9d48fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:58.349656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1424/2843] 10.222.0.93 () {58 vars in 967 bytes} [Fri Jun 3 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:58.366357 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136713e-b8ca-427b-b3d6-ec4fa7fe8488 service swift] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:58.378198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136713e-b8ca-427b-b3d6-ec4fa7fe8488 service swift] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:58.385634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e5f56-56c4-4baf-ad37-c391bb8bbff5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:58.387115 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e5f56-56c4-4baf-ad37-c391bb8bbff5 service swift] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:58.387647 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e5f56-56c4-4baf-ad37-c391bb8bbff5 service swift] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:58.388071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e5f56-56c4-4baf-ad37-c391bb8bbff5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:58.398861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e5f56-56c4-4baf-ad37-c391bb8bbff5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:58.400514 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e5f56-56c4-4baf-ad37-c391bb8bbff5 service swift] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:58.411241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1420/2844] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:35:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 03 15:35:58.704277 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9567f-9593-407d-b889-f8dd48373fe4 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:58.704779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9567f-9593-407d-b889-f8dd48373fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:58.704867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c9567f-9593-407d-b889-f8dd48373fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:58.726119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-60c9567f-9593-407d-b889-f8dd48373fe4 None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:58.727812 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-60c9567f-9593-407d-b889-f8dd48373fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:58.737108 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1425/2845] 10.222.0.93 () {58 vars in 1009 bytes} [Fri Jun 3 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:58.756787 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e5f56-56c4-4baf-ad37-c391bb8bbff5 service swift] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:58.768484 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e5f56-56c4-4baf-ad37-c391bb8bbff5 service swift] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:58.775085 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23aa8b8-bc27-4f3b-b6dc-22d56720822c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:58.775831 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23aa8b8-bc27-4f3b-b6dc-22d56720822c service swift] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:58.776353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23aa8b8-bc27-4f3b-b6dc-22d56720822c service swift] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:58.776850 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23aa8b8-bc27-4f3b-b6dc-22d56720822c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:58.787008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23aa8b8-bc27-4f3b-b6dc-22d56720822c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:58.788950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23aa8b8-bc27-4f3b-b6dc-22d56720822c service swift] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:58.798637 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1421/2846] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:35:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 03 15:35:59.223987 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64b6f-959e-4307-ac5c-a4efd7241c0f None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:59.224324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64b6f-959e-4307-ac5c-a4efd7241c0f None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:59.224705 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e64b6f-959e-4307-ac5c-a4efd7241c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:59.248389 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-e8e64b6f-959e-4307-ac5c-a4efd7241c0f None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:35:59.249912 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e8e64b6f-959e-4307-ac5c-a4efd7241c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:35:59.261945 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1426/2847] 10.222.0.93 () {58 vars in 967 bytes} [Fri Jun 3 15:35:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:35:59.278067 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23aa8b8-bc27-4f3b-b6dc-22d56720822c service swift] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:35:59.289822 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23aa8b8-bc27-4f3b-b6dc-22d56720822c service swift] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:35:59.295846 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9b6f9-2f2a-4ff2-9fed-324b24f42243 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:35:59.296497 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9b6f9-2f2a-4ff2-9fed-324b24f42243 service swift] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:35:59.296779 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9b6f9-2f2a-4ff2-9fed-324b24f42243 service swift] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:35:59.297055 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9b6f9-2f2a-4ff2-9fed-324b24f42243 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:35:59.307110 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9b6f9-2f2a-4ff2-9fed-324b24f42243 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:35:59.308939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9b6f9-2f2a-4ff2-9fed-324b24f42243 service swift] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:35:59.322406 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1422/2848] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:35:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:36:01.653568 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e64b6f-959e-4307-ac5c-a4efd7241c0f None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:01.655746 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e8e64b6f-959e-4307-ac5c-a4efd7241c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:01.751079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e64b6f-959e-4307-ac5c-a4efd7241c0f None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:01.755844 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:01.756730 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:01.757015 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:01.757290 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] PATH_INFO: `/v3/users/134977bf87a2409481cf99d98c8b2228` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:01.758870 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=134977bf87a2409481cf99d98c8b2228)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:01.760564 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:01.820406 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:01.838491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '134977bf87a2409481cf99d98c8b2228'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:01.859866 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '134977bf87a2409481cf99d98c8b2228'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:01.861366 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '134977bf87a2409481cf99d98c8b2228'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:01.873011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '134977bf87a2409481cf99d98c8b2228'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:01.874090 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '134977bf87a2409481cf99d98c8b2228'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:01.893593 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1427/2849] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:36:01 2022] DELETE /identity/v3/users/134977bf87a2409481cf99d98c8b2228 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:01.900517 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fdb42-e3be-43dd-a150-3f3e9a7ad0a4 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:01.901614 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fdb42-e3be-43dd-a150-3f3e9a7ad0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:01.904053 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fdb42-e3be-43dd-a150-3f3e9a7ad0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:01.927815 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-e42fdb42-e3be-43dd-a150-3f3e9a7ad0a4 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:36:01.929095 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e42fdb42-e3be-43dd-a150-3f3e9a7ad0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:02.044970 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1423/2850] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:36:02.059984 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:02.061852 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:02.129949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a381a78-f2d1-464a-a2c3-1238b0f37a77 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:02.139309 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c6bc1-068e-456f-b5cf-053231561b8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:02.140079 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c6bc1-068e-456f-b5cf-053231561b8e service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:02.140418 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c6bc1-068e-456f-b5cf-053231561b8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:02.141099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c6bc1-068e-456f-b5cf-053231561b8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:02.155902 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c6bc1-068e-456f-b5cf-053231561b8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:02.157389 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c6bc1-068e-456f-b5cf-053231561b8e service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:02.176543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1428/2851] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:36:02.503166 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fdb42-e3be-43dd-a150-3f3e9a7ad0a4 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:02.505939 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-e42fdb42-e3be-43dd-a150-3f3e9a7ad0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:02.552842 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fdb42-e3be-43dd-a150-3f3e9a7ad0a4 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:02.560468 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:02.564015 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:02.564396 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:02.564708 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] PATH_INFO: `/v3/projects/acf00e4ef6f649e69980d6ec2c4b8bea` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:02.567463 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf00e4ef6f649e69980d6ec2c4b8bea)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:02.569515 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:02.593445 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:02.655600 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf00e4ef6f649e69980d6ec2c4b8bea', 'request_id': 'req-3670d2bf-fa24-47cc-b1b7-d58c73e21787'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:02.664882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf00e4ef6f649e69980d6ec2c4b8bea', 'request_id': 'req-3670d2bf-fa24-47cc-b1b7-d58c73e21787'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:02.682479 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf00e4ef6f649e69980d6ec2c4b8bea', 'request_id': 'req-3670d2bf-fa24-47cc-b1b7-d58c73e21787'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:02.703662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] The token cache is being invalidate because project acf00e4ef6f649e69980d6ec2c4b8bea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:36:02.704241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:02.705241 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:02.710574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1424/2852] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:36:02 2022] DELETE /identity/v3/projects/acf00e4ef6f649e69980d6ec2c4b8bea => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:02.812313 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c6bc1-068e-456f-b5cf-053231561b8e service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:02.816158 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-830c6bc1-068e-456f-b5cf-053231561b8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:02.846508 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c6bc1-068e-456f-b5cf-053231561b8e service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:02.937846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a0265-78fd-473f-89ae-477b5b80d356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:02.939305 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a0265-78fd-473f-89ae-477b5b80d356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:02.939828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a0265-78fd-473f-89ae-477b5b80d356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:02.940333 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a0265-78fd-473f-89ae-477b5b80d356 admin admin] PATH_INFO: `/v3/users/5b82f76d685041d084dd5f332a5b1105` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:02.941813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a0265-78fd-473f-89ae-477b5b80d356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b82f76d685041d084dd5f332a5b1105)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:02.944408 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a0265-78fd-473f-89ae-477b5b80d356 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:03.016941 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7c0a0265-78fd-473f-89ae-477b5b80d356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b82f76d685041d084dd5f332a5b1105'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:03.021801 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:03.034831 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7c0a0265-78fd-473f-89ae-477b5b80d356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b82f76d685041d084dd5f332a5b1105'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:03.036247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7c0a0265-78fd-473f-89ae-477b5b80d356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b82f76d685041d084dd5f332a5b1105'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:03.047521 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7c0a0265-78fd-473f-89ae-477b5b80d356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b82f76d685041d084dd5f332a5b1105'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:03.048246 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7c0a0265-78fd-473f-89ae-477b5b80d356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b82f76d685041d084dd5f332a5b1105'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:03.074081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1429/2853] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:36:02 2022] DELETE /identity/v3/users/5b82f76d685041d084dd5f332a5b1105 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:03.288226 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:03.290638 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:03.308380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3670d2bf-fa24-47cc-b1b7-d58c73e21787 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:03.387438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:03.388316 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:03.388597 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:03.388869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] PATH_INFO: `/v3/projects/d5151421b3ee4477924cdab41025075b` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:03.390693 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5151421b3ee4477924cdab41025075b)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:03.392665 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:03.443099 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5151421b3ee4477924cdab41025075b', 'request_id': 'req-58b92715-3504-4531-9213-b13f65e74237'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:03.450546 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5151421b3ee4477924cdab41025075b', 'request_id': 'req-58b92715-3504-4531-9213-b13f65e74237'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:03.459266 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5151421b3ee4477924cdab41025075b', 'request_id': 'req-58b92715-3504-4531-9213-b13f65e74237'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:03.471574 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] The token cache is being invalidate because project d5151421b3ee4477924cdab41025075b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:36:03.471958 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:03.472668 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-58b92715-3504-4531-9213-b13f65e74237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:03.474274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1425/2854] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:36:03 2022] DELETE /identity/v3/projects/d5151421b3ee4477924cdab41025075b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:03.479350 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f38b0-3fdb-463f-ad20-2e7ff5bd6640 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:03.479717 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f38b0-3fdb-463f-ad20-2e7ff5bd6640 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:03.479998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f38b0-3fdb-463f-ad20-2e7ff5bd6640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:03.500615 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-348f38b0-3fdb-463f-ad20-2e7ff5bd6640 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:36:03.501867 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-348f38b0-3fdb-463f-ad20-2e7ff5bd6640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:03.571623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1430/2855] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:36:03.579477 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e24da-d597-4785-b4e4-42f7af22d48f None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:03.579764 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e24da-d597-4785-b4e4-42f7af22d48f None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:03.580045 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e24da-d597-4785-b4e4-42f7af22d48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:03.599021 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-6b7e24da-d597-4785-b4e4-42f7af22d48f None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:36:03.600344 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-6b7e24da-d597-4785-b4e4-42f7af22d48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:03.608896 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1426/2856] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:36:03.613179 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f38b0-3fdb-463f-ad20-2e7ff5bd6640 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:03.624053 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f38b0-3fdb-463f-ad20-2e7ff5bd6640 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:03.628785 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b114d-8076-4de8-9ad6-8f451981dd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:03.629529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b114d-8076-4de8-9ad6-8f451981dd65 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:03.629879 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b114d-8076-4de8-9ad6-8f451981dd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:03.630225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b114d-8076-4de8-9ad6-8f451981dd65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:03.631086 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b114d-8076-4de8-9ad6-8f451981dd65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:36:03.631461 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b114d-8076-4de8-9ad6-8f451981dd65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:03.633126 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b114d-8076-4de8-9ad6-8f451981dd65 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:03.646569 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1431/2857] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:36:03.652006 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e24da-d597-4785-b4e4-42f7af22d48f None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:03.661634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e24da-d597-4785-b4e4-42f7af22d48f None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:03.667213 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f4578-31a8-4e5e-b5e1-6a2dad6aa97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:03.668216 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f4578-31a8-4e5e-b5e1-6a2dad6aa97f admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:03.668659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f4578-31a8-4e5e-b5e1-6a2dad6aa97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:03.668911 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f4578-31a8-4e5e-b5e1-6a2dad6aa97f admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:03.669605 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f4578-31a8-4e5e-b5e1-6a2dad6aa97f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:03.672741 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f4578-31a8-4e5e-b5e1-6a2dad6aa97f admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:03.701610 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1427/2858] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:36:03 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:36:03.705781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b114d-8076-4de8-9ad6-8f451981dd65 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:03.715856 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b114d-8076-4de8-9ad6-8f451981dd65 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:03.771229 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94db10-58e7-447e-a884-90559d4719a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:03.771892 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94db10-58e7-447e-a884-90559d4719a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:03.772175 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94db10-58e7-447e-a884-90559d4719a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:03.772447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a94db10-58e7-447e-a884-90559d4719a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:03.773135 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94db10-58e7-447e-a884-90559d4719a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:03.775102 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a94db10-58e7-447e-a884-90559d4719a4 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:03.795846 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1432/2859] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:36:03 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:36:03.800023 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f4578-31a8-4e5e-b5e1-6a2dad6aa97f admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:03.811008 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f4578-31a8-4e5e-b5e1-6a2dad6aa97f admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:03.815621 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e6d9e-235f-47f8-a55e-2898c7675675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:03.816273 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6d9e-235f-47f8-a55e-2898c7675675 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:03.816555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6d9e-235f-47f8-a55e-2898c7675675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:03.816829 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6d9e-235f-47f8-a55e-2898c7675675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:03.817552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e6d9e-235f-47f8-a55e-2898c7675675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:03.819179 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e6d9e-235f-47f8-a55e-2898c7675675 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:03.820438 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:03.831959 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1428/2860] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 03 15:36:03.835956 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94db10-58e7-447e-a884-90559d4719a4 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:03.846507 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a94db10-58e7-447e-a884-90559d4719a4 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:03.851139 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b630f-0725-435d-a217-33f46616917e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:03.851795 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b630f-0725-435d-a217-33f46616917e admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:03.852081 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b630f-0725-435d-a217-33f46616917e admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:03.852367 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b630f-0725-435d-a217-33f46616917e admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:03.853090 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b630f-0725-435d-a217-33f46616917e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:03.854741 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b630f-0725-435d-a217-33f46616917e admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:03.866254 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1433/2861] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:36:03.870353 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e6d9e-235f-47f8-a55e-2898c7675675 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:03.880212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e6d9e-235f-47f8-a55e-2898c7675675 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:03.884889 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c411d1-5d89-4a0d-8435-2a0bed63cb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:03.885756 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c411d1-5d89-4a0d-8435-2a0bed63cb0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:03.886034 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c411d1-5d89-4a0d-8435-2a0bed63cb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:03.886306 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c411d1-5d89-4a0d-8435-2a0bed63cb0e admin admin] PATH_INFO: `/v3/projects/d4fde2612ff24c67a8b060635dbd3917/users/fe0b738f860c4c018b2719e61c4a7b54/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:03.901024 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c411d1-5d89-4a0d-8435-2a0bed63cb0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4fde2612ff24c67a8b060635dbd3917, user_id=fe0b738f860c4c018b2719e61c4a7b54, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:03.903784 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c411d1-5d89-4a0d-8435-2a0bed63cb0e admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:03.917634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1429/2862] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:36:03 2022] PUT /identity/v3/projects/d4fde2612ff24c67a8b060635dbd3917/users/fe0b738f860c4c018b2719e61c4a7b54/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:03.928141 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984f911-ef53-4a8e-b208-dbdaa0af0ce0 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:03.928503 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984f911-ef53-4a8e-b208-dbdaa0af0ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:03.928854 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984f911-ef53-4a8e-b208-dbdaa0af0ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:03.956612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-0984f911-ef53-4a8e-b208-dbdaa0af0ce0 None None] MFA Rules not processed for user `fe0b738f860c4c018b2719e61c4a7b54`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:36:03.957817 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0984f911-ef53-4a8e-b208-dbdaa0af0ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:03.984034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:04.024549 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1434/2863] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:36:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:36:04.035675 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c411d1-5d89-4a0d-8435-2a0bed63cb0e admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:04.037381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f7c411d1-5d89-4a0d-8435-2a0bed63cb0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:04.094755 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c411d1-5d89-4a0d-8435-2a0bed63cb0e admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:04.098864 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32487e-2002-4a42-85cd-74a3d7656ba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:04.099507 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32487e-2002-4a42-85cd-74a3d7656ba4 service cinder] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:04.099788 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32487e-2002-4a42-85cd-74a3d7656ba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:04.100465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32487e-2002-4a42-85cd-74a3d7656ba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:04.109724 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32487e-2002-4a42-85cd-74a3d7656ba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:04.111274 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32487e-2002-4a42-85cd-74a3d7656ba4 service cinder] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:04.120071 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1430/2864] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:36:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:36:04.195279 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46fcda-0fd6-439b-be52-980667b94074 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:04.195651 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46fcda-0fd6-439b-be52-980667b94074 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:04.196012 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46fcda-0fd6-439b-be52-980667b94074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:04.214566 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-2a46fcda-0fd6-439b-be52-980667b94074 None None] MFA Rules not processed for user `0603019b08d94980863f6612907601a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:36:04.215794 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2a46fcda-0fd6-439b-be52-980667b94074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:04.277653 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1435/2865] 10.222.0.93 () {58 vars in 967 bytes} [Fri Jun 3 15:36:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:36:04.290704 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32487e-2002-4a42-85cd-74a3d7656ba4 service cinder] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:04.292288 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-7e32487e-2002-4a42-85cd-74a3d7656ba4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:04.377994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32487e-2002-4a42-85cd-74a3d7656ba4 service cinder] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:04.383428 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d00a7-16a7-4ca7-95b5-8e60c5acf617 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb4fb8f859dc4489b8c4fd8fdc09c3f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:04.384207 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d00a7-16a7-4ca7-95b5-8e60c5acf617 service swift] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:04.384568 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d00a7-16a7-4ca7-95b5-8e60c5acf617 service swift] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:04.384913 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d00a7-16a7-4ca7-95b5-8e60c5acf617 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:04.395584 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d00a7-16a7-4ca7-95b5-8e60c5acf617 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:04.397195 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d00a7-16a7-4ca7-95b5-8e60c5acf617 service swift] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:04.406767 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1431/2866] 10.222.0.93 () {60 vars in 1384 bytes} [Fri Jun 3 15:36:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 03 15:36:06.525998 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46fcda-0fd6-439b-be52-980667b94074 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:06.527593 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-2a46fcda-0fd6-439b-be52-980667b94074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:06.597783 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46fcda-0fd6-439b-be52-980667b94074 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:06.602491 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb9359-4304-4778-89d2-da8ee56be86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:06.603402 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb9359-4304-4778-89d2-da8ee56be86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:06.603686 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb9359-4304-4778-89d2-da8ee56be86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:06.603958 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb9359-4304-4778-89d2-da8ee56be86b admin admin] PATH_INFO: `/v3/users/4db856168c34491080b93c8ee5680c2a` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:06.605294 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb9359-4304-4778-89d2-da8ee56be86b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db856168c34491080b93c8ee5680c2a)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:06.607264 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb9359-4304-4778-89d2-da8ee56be86b admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:06.662824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b1bb9359-4304-4778-89d2-da8ee56be86b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db856168c34491080b93c8ee5680c2a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:06.679800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b1bb9359-4304-4778-89d2-da8ee56be86b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db856168c34491080b93c8ee5680c2a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:06.680519 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b1bb9359-4304-4778-89d2-da8ee56be86b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db856168c34491080b93c8ee5680c2a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:06.685708 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:06.687843 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b1bb9359-4304-4778-89d2-da8ee56be86b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db856168c34491080b93c8ee5680c2a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:06.688770 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-b1bb9359-4304-4778-89d2-da8ee56be86b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db856168c34491080b93c8ee5680c2a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:06.702300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1436/2867] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:36:06 2022] DELETE /identity/v3/users/4db856168c34491080b93c8ee5680c2a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:06.873994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d00a7-16a7-4ca7-95b5-8e60c5acf617 service swift] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:06.885009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d00a7-16a7-4ca7-95b5-8e60c5acf617 service swift] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:06.943295 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:06.944286 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:06.944566 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:06.944841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] PATH_INFO: `/v3/projects/bcb5131601b14b5884956a189e6e9462` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:06.946017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb5131601b14b5884956a189e6e9462)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:06.947950 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:06.954641 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:06.994197 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb5131601b14b5884956a189e6e9462', 'request_id': 'req-14390d6c-648b-4bb5-a848-bd6d5158788c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:07.000119 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb5131601b14b5884956a189e6e9462', 'request_id': 'req-14390d6c-648b-4bb5-a848-bd6d5158788c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:07.007058 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb5131601b14b5884956a189e6e9462', 'request_id': 'req-14390d6c-648b-4bb5-a848-bd6d5158788c'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:07.018794 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] The token cache is being invalidate because project bcb5131601b14b5884956a189e6e9462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:36:07.019077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:07.019692 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-14390d6c-648b-4bb5-a848-bd6d5158788c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:07.021212 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1432/2868] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:36:06 2022] DELETE /identity/v3/projects/bcb5131601b14b5884956a189e6e9462 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:07.026109 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d8974-4fc9-4199-b681-86fc52778103 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:07.026397 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d8974-4fc9-4199-b681-86fc52778103 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:07.026707 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d8974-4fc9-4199-b681-86fc52778103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:07.044299 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-830d8974-4fc9-4199-b681-86fc52778103 None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:36:07.045485 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-830d8974-4fc9-4199-b681-86fc52778103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:07.103195 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1437/2869] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:36:07.111634 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2510c8e-25b0-4383-bd93-e00428e36dcd None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:07.111931 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2510c8e-25b0-4383-bd93-e00428e36dcd None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:07.112199 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2510c8e-25b0-4383-bd93-e00428e36dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:07.130772 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-f2510c8e-25b0-4383-bd93-e00428e36dcd None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:36:07.131967 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-f2510c8e-25b0-4383-bd93-e00428e36dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:07.140629 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1433/2870] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:36:07.144799 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d8974-4fc9-4199-b681-86fc52778103 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:07.154724 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d8974-4fc9-4199-b681-86fc52778103 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:07.159287 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fdc92-50fb-41a8-8dcc-89051b688e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:07.159951 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fdc92-50fb-41a8-8dcc-89051b688e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:07.160235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fdc92-50fb-41a8-8dcc-89051b688e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:07.160514 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fdc92-50fb-41a8-8dcc-89051b688e14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:07.161235 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fdc92-50fb-41a8-8dcc-89051b688e14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80222) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 03 15:36:07.161544 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fdc92-50fb-41a8-8dcc-89051b688e14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:07.163196 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fdc92-50fb-41a8-8dcc-89051b688e14 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:07.175349 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1438/2871] 10.222.0.93 () {58 vars in 1152 bytes} [Fri Jun 3 15:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 03 15:36:07.179802 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2510c8e-25b0-4383-bd93-e00428e36dcd None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:07.190748 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2510c8e-25b0-4383-bd93-e00428e36dcd None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:07.195938 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c84ee4-297b-49b6-ba1c-b60063085ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:07.196591 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c84ee4-297b-49b6-ba1c-b60063085ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:07.196869 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c84ee4-297b-49b6-ba1c-b60063085ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:07.197167 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c84ee4-297b-49b6-ba1c-b60063085ce8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:07.197861 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c84ee4-297b-49b6-ba1c-b60063085ce8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:07.199809 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c84ee4-297b-49b6-ba1c-b60063085ce8 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:07.224607 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1434/2872] 10.222.0.93 () {60 vars in 1152 bytes} [Fri Jun 3 15:36:07 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:36:07.228824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fdc92-50fb-41a8-8dcc-89051b688e14 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:07.239312 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fdc92-50fb-41a8-8dcc-89051b688e14 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:07.296137 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdbc7a-0a74-45f8-b455-61dc2c512065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:07.297119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdbc7a-0a74-45f8-b455-61dc2c512065 admin admin] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:07.297119 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdbc7a-0a74-45f8-b455-61dc2c512065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:07.297399 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bdbc7a-0a74-45f8-b455-61dc2c512065 admin admin] PATH_INFO: `/v3/users` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:07.298084 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdbc7a-0a74-45f8-b455-61dc2c512065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:07.300065 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bdbc7a-0a74-45f8-b455-61dc2c512065 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:07.320523 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1439/2873] 10.222.0.93 () {60 vars in 1143 bytes} [Fri Jun 3 15:36:07 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 03 15:36:07.324625 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c84ee4-297b-49b6-ba1c-b60063085ce8 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:07.334999 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c84ee4-297b-49b6-ba1c-b60063085ce8 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:07.339557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a1916-1af2-4eab-938a-2f516393edc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:07.340211 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a1916-1af2-4eab-938a-2f516393edc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:07.340494 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a1916-1af2-4eab-938a-2f516393edc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:07.340770 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75a1916-1af2-4eab-938a-2f516393edc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:07.341491 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a1916-1af2-4eab-938a-2f516393edc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:07.343121 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75a1916-1af2-4eab-938a-2f516393edc3 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:07.354283 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1435/2874] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:36:07.358572 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdbc7a-0a74-45f8-b455-61dc2c512065 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:07.368526 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bdbc7a-0a74-45f8-b455-61dc2c512065 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:07.373122 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb850c6-1dc6-4e3e-8e58-e1a39f044a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:07.374474 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb850c6-1dc6-4e3e-8e58-e1a39f044a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:07.374781 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb850c6-1dc6-4e3e-8e58-e1a39f044a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:07.375067 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb850c6-1dc6-4e3e-8e58-e1a39f044a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:07.375790 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb850c6-1dc6-4e3e-8e58-e1a39f044a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:07.377386 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb850c6-1dc6-4e3e-8e58-e1a39f044a09 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:07.387567 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1440/2875] 10.222.0.93 () {58 vars in 1121 bytes} [Fri Jun 3 15:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 03 15:36:07.391489 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a1916-1af2-4eab-938a-2f516393edc3 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:07.401707 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75a1916-1af2-4eab-938a-2f516393edc3 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:07.406324 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d9110-69c4-4024-a340-db9943364591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:07.407280 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d9110-69c4-4024-a340-db9943364591 admin admin] REQUEST_METHOD: `PUT` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:07.407555 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d9110-69c4-4024-a340-db9943364591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:07.407888 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d9110-69c4-4024-a340-db9943364591 admin admin] PATH_INFO: `/v3/projects/356746cc3b404ce1838d403eea4cb3b5/users/13aec7799de64fdd89257dc377b515a7/roles/1249af5cf5054410a11b2d88c03fcd52` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:07.422897 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d9110-69c4-4024-a340-db9943364591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=356746cc3b404ce1838d403eea4cb3b5, user_id=13aec7799de64fdd89257dc377b515a7, role_id=1249af5cf5054410a11b2d88c03fcd52)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:07.425518 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22d9110-69c4-4024-a340-db9943364591 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:07.439340 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1436/2876] 10.222.0.93 () {60 vars in 1482 bytes} [Fri Jun 3 15:36:07 2022] PUT /identity/v3/projects/356746cc3b404ce1838d403eea4cb3b5/users/13aec7799de64fdd89257dc377b515a7/roles/1249af5cf5054410a11b2d88c03fcd52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:07.449283 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e48e3-51bb-49a8-892e-e6bd2b3fa601 None None] REQUEST_METHOD: `POST` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:07.449573 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e48e3-51bb-49a8-892e-e6bd2b3fa601 None None] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:07.449878 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e48e3-51bb-49a8-892e-e6bd2b3fa601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:07.475680 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.auth.core [None req-c74e48e3-51bb-49a8-892e-e6bd2b3fa601 None None] MFA Rules not processed for user `13aec7799de64fdd89257dc377b515a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80222) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:36:07.476747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c74e48e3-51bb-49a8-892e-e6bd2b3fa601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:07.531545 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:07.569877 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1441/2877] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:36:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:36:07.582091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d9110-69c4-4024-a340-db9943364591 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:07.583832 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-c22d9110-69c4-4024-a340-db9943364591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:07.648038 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d9110-69c4-4024-a340-db9943364591 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:07.651791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a325613-e7c7-4e23-a49e-c463331cf207 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13121a263c31433dbd4a5082b4bf3bb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:07.652442 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a325613-e7c7-4e23-a49e-c463331cf207 service glance] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:07.652726 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a325613-e7c7-4e23-a49e-c463331cf207 service glance] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:07.653003 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a325613-e7c7-4e23-a49e-c463331cf207 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:07.663009 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a325613-e7c7-4e23-a49e-c463331cf207 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:07.664479 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a325613-e7c7-4e23-a49e-c463331cf207 service glance] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:07.675172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1437/2878] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:36:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:36:17.565197 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=7 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:17.669973 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:22.391543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e48e3-51bb-49a8-892e-e6bd2b3fa601 None None] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:22.393002 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c74e48e3-51bb-49a8-892e-e6bd2b3fa601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:22.461983 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e48e3-51bb-49a8-892e-e6bd2b3fa601 None None] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:22.466676 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:22.467638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:22.468099 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:22.468381 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] PATH_INFO: `/v3/users/fe0b738f860c4c018b2719e61c4a7b54` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:22.469609 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0b738f860c4c018b2719e61c4a7b54)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:22.471612 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:22.525862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0b738f860c4c018b2719e61c4a7b54'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:22.540784 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0b738f860c4c018b2719e61c4a7b54'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:22.541542 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0b738f860c4c018b2719e61c4a7b54'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:22.549828 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0b738f860c4c018b2719e61c4a7b54'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:22.550780 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0b738f860c4c018b2719e61c4a7b54'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:22.565204 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1442/2879] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:36:22 2022] DELETE /identity/v3/users/fe0b738f860c4c018b2719e61c4a7b54 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:22.569929 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120387b5-6b82-4d3e-b305-db66f2e2532a None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:22.570427 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120387b5-6b82-4d3e-b305-db66f2e2532a None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:22.570552 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120387b5-6b82-4d3e-b305-db66f2e2532a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:22.588714 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-120387b5-6b82-4d3e-b305-db66f2e2532a None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:36:22.589930 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-120387b5-6b82-4d3e-b305-db66f2e2532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:22.658146 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1438/2880] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:36:22.668274 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:22.669732 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:22.733665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292d2ba-b150-4906-8f3d-5b95ebb39d90 admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:22.737779 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86f332-a1dc-43a3-af1b-f311b9a3a599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:22.738430 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86f332-a1dc-43a3-af1b-f311b9a3a599 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:22.738739 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86f332-a1dc-43a3-af1b-f311b9a3a599 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:22.739013 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86f332-a1dc-43a3-af1b-f311b9a3a599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:22.750424 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86f332-a1dc-43a3-af1b-f311b9a3a599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:22.752116 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86f332-a1dc-43a3-af1b-f311b9a3a599 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:22.762764 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1443/2881] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:36:23.049865 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120387b5-6b82-4d3e-b305-db66f2e2532a None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:23.051743 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-120387b5-6b82-4d3e-b305-db66f2e2532a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:23.066841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120387b5-6b82-4d3e-b305-db66f2e2532a None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:23.073571 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:23.074570 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:23.074980 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:23.075342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] PATH_INFO: `/v3/projects/d4fde2612ff24c67a8b060635dbd3917` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:23.076662 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4fde2612ff24c67a8b060635dbd3917)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:23.078760 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:23.136219 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4fde2612ff24c67a8b060635dbd3917', 'request_id': 'req-ce64f14a-0ce0-49f9-b522-5a390c962a98'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:23.142343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4fde2612ff24c67a8b060635dbd3917', 'request_id': 'req-ce64f14a-0ce0-49f9-b522-5a390c962a98'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:23.150374 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4fde2612ff24c67a8b060635dbd3917', 'request_id': 'req-ce64f14a-0ce0-49f9-b522-5a390c962a98'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:23.162729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] The token cache is being invalidate because project d4fde2612ff24c67a8b060635dbd3917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:36:23.162943 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:23.163606 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-ce64f14a-0ce0-49f9-b522-5a390c962a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:23.165265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1439/2882] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:36:23 2022] DELETE /identity/v3/projects/d4fde2612ff24c67a8b060635dbd3917 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:32.757678 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:33.159520 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:39.798181 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86f332-a1dc-43a3-af1b-f311b9a3a599 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:39.799657 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-1b86f332-a1dc-43a3-af1b-f311b9a3a599 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:39.812120 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86f332-a1dc-43a3-af1b-f311b9a3a599 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:39.866891 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:39.867753 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:39.868036 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:39.868310 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] PATH_INFO: `/v3/users/13aec7799de64fdd89257dc377b515a7` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:39.869489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=13aec7799de64fdd89257dc377b515a7)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:39.871446 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:39.917359 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13aec7799de64fdd89257dc377b515a7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:39.929670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13aec7799de64fdd89257dc377b515a7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:39.930273 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13aec7799de64fdd89257dc377b515a7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:39.937032 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13aec7799de64fdd89257dc377b515a7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:39.937656 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13aec7799de64fdd89257dc377b515a7'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:39.950775 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1444/2883] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:36:39 2022] DELETE /identity/v3/users/13aec7799de64fdd89257dc377b515a7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:39.955290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833e98-e63c-4216-8916-d9059c83b4ab None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:39.955789 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833e98-e63c-4216-8916-d9059c83b4ab None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:39.955974 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5833e98-e63c-4216-8916-d9059c83b4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:39.973750 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-d5833e98-e63c-4216-8916-d9059c83b4ab None None] MFA Rules not processed for user `b868f066d8ff4579b72df10988c4ede1`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:36:39.975277 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d5833e98-e63c-4216-8916-d9059c83b4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:40.033087 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1440/2884] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 03 15:36:40.042670 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:40.044071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:40.100189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e306780-8d3e-47e2-9dfa-ab930c234bdd admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:40.103990 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32594e2f-69df-4daa-81ff-2951804354b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:40.104638 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594e2f-69df-4daa-81ff-2951804354b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:40.104925 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594e2f-69df-4daa-81ff-2951804354b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:40.105201 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32594e2f-69df-4daa-81ff-2951804354b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:40.114896 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32594e2f-69df-4daa-81ff-2951804354b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:40.116340 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32594e2f-69df-4daa-81ff-2951804354b4 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:40.125657 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1445/2885] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:36:40.394873 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5833e98-e63c-4216-8916-d9059c83b4ab None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:36:40.396290 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-d5833e98-e63c-4216-8916-d9059c83b4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:36:40.409810 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5833e98-e63c-4216-8916-d9059c83b4ab None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:36:40.414862 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:36:40.416017 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:36:40.416369 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:36:40.416651 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] PATH_INFO: `/v3/projects/356746cc3b404ce1838d403eea4cb3b5` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:36:40.417998 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=356746cc3b404ce1838d403eea4cb3b5)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:36:40.419997 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:36:40.466101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '356746cc3b404ce1838d403eea4cb3b5', 'request_id': 'req-15827ca8-12e9-4142-a73f-dc6af53f3503'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:40.472250 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '356746cc3b404ce1838d403eea4cb3b5', 'request_id': 'req-15827ca8-12e9-4142-a73f-dc6af53f3503'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:40.479805 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '356746cc3b404ce1838d403eea4cb3b5', 'request_id': 'req-15827ca8-12e9-4142-a73f-dc6af53f3503'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:40.491177 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] The token cache is being invalidate because project 356746cc3b404ce1838d403eea4cb3b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80221) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:36:40.491465 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:40.492101 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-15827ca8-12e9-4142-a73f-dc6af53f3503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:36:40.493630 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1441/2886] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:36:40 2022] DELETE /identity/v3/projects/356746cc3b404ce1838d403eea4cb3b5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:36:50.121505 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:36:50.488138 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:37:15.388423 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32594e2f-69df-4daa-81ff-2951804354b4 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:37:15.389820 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-32594e2f-69df-4daa-81ff-2951804354b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:15.449744 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32594e2f-69df-4daa-81ff-2951804354b4 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:37:15.496268 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4240595-1bc0-4eb8-a8da-dd9869ceeeee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:37:15.496907 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4240595-1bc0-4eb8-a8da-dd9869ceeeee service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:37:15.497189 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4240595-1bc0-4eb8-a8da-dd9869ceeeee service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:37:15.497464 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4240595-1bc0-4eb8-a8da-dd9869ceeeee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:37:15.499247 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d4240595-1bc0-4eb8-a8da-dd9869ceeeee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:15.557829 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4240595-1bc0-4eb8-a8da-dd9869ceeeee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:37:15.559300 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4240595-1bc0-4eb8-a8da-dd9869ceeeee service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:37:15.568186 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1446/2887] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:37:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:37:23.419259 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94612e2b-0e37-48a0-a05a-ee9fc202fb60 None None] REQUEST_METHOD: `POST` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:37:23.419771 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94612e2b-0e37-48a0-a05a-ee9fc202fb60 None None] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:37:23.419900 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94612e2b-0e37-48a0-a05a-ee9fc202fb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:37:23.438203 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.auth.core [None req-94612e2b-0e37-48a0-a05a-ee9fc202fb60 None None] MFA Rules not processed for user `7d53405e3f88439f81b67106ad98adc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80221) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 03 15:37:23.439434 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-94612e2b-0e37-48a0-a05a-ee9fc202fb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:23.446994 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1442/2888] 10.222.0.93 () {58 vars in 957 bytes} [Fri Jun 3 15:37:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 03 15:37:23.485456 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4240595-1bc0-4eb8-a8da-dd9869ceeeee service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:37:23.487549 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d4240595-1bc0-4eb8-a8da-dd9869ceeeee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:23.498071 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4240595-1bc0-4eb8-a8da-dd9869ceeeee service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:37:23.501688 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28eff-c91c-43eb-80f2-e56ed804f3ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98aa51c4bc604b0ca8a6c69bc1455bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:37:23.502318 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28eff-c91c-43eb-80f2-e56ed804f3ed service cinder] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:37:23.502631 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28eff-c91c-43eb-80f2-e56ed804f3ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:37:23.502909 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b28eff-c91c-43eb-80f2-e56ed804f3ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:37:23.504591 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f7b28eff-c91c-43eb-80f2-e56ed804f3ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:23.572725 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=100 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:37:23.575575 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b28eff-c91c-43eb-80f2-e56ed804f3ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:37:23.577346 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b28eff-c91c-43eb-80f2-e56ed804f3ed service cinder] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:37:23.589738 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1447/2889] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:37:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:37:24.315581 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94612e2b-0e37-48a0-a05a-ee9fc202fb60 None None] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:37:24.316983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-94612e2b-0e37-48a0-a05a-ee9fc202fb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:24.324963 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28eff-c91c-43eb-80f2-e56ed804f3ed service cinder] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:37:24.327808 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-f7b28eff-c91c-43eb-80f2-e56ed804f3ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:24.386809 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b28eff-c91c-43eb-80f2-e56ed804f3ed service cinder] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:37:24.388209 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94612e2b-0e37-48a0-a05a-ee9fc202fb60 None None] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:37:24.390637 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4144188-5520-4b96-b2e7-57ac577589b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:37:24.391272 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4144188-5520-4b96-b2e7-57ac577589b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:37:24.391551 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4144188-5520-4b96-b2e7-57ac577589b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:37:24.391826 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4144188-5520-4b96-b2e7-57ac577589b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:37:24.393912 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881df572-1d13-4bce-862b-6f5fb252d8d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:37:24.394091 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-d4144188-5520-4b96-b2e7-57ac577589b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:24.394627 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881df572-1d13-4bce-862b-6f5fb252d8d1 service placement] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:37:24.394971 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881df572-1d13-4bce-862b-6f5fb252d8d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:37:24.395255 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881df572-1d13-4bce-862b-6f5fb252d8d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:37:24.397019 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-881df572-1d13-4bce-862b-6f5fb252d8d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:24.407256 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4144188-5520-4b96-b2e7-57ac577589b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:37:24.409225 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4144188-5520-4b96-b2e7-57ac577589b7 service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:37:24.410343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881df572-1d13-4bce-862b-6f5fb252d8d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:37:24.411976 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881df572-1d13-4bce-862b-6f5fb252d8d1 service placement] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:37:24.420042 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1448/2890] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:37:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 03 15:37:24.422246 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1443/2891] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:37:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:37:26.094543 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4144188-5520-4b96-b2e7-57ac577589b7 service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:37:26.104145 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4144188-5520-4b96-b2e7-57ac577589b7 service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:37:26.108124 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6998667-5cdc-4bd4-ac30-13015b5802da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:37:26.108862 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6998667-5cdc-4bd4-ac30-13015b5802da service neutron] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:37:26.109146 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6998667-5cdc-4bd4-ac30-13015b5802da service neutron] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:37:26.109422 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6998667-5cdc-4bd4-ac30-13015b5802da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:37:26.111515 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-c6998667-5cdc-4bd4-ac30-13015b5802da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:26.122240 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6998667-5cdc-4bd4-ac30-13015b5802da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:37:26.123859 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6998667-5cdc-4bd4-ac30-13015b5802da service neutron] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:37:26.132601 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1449/2892] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:37:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:37:26.863649 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881df572-1d13-4bce-862b-6f5fb252d8d1 service placement] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:37:26.865198 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-881df572-1d13-4bce-862b-6f5fb252d8d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:26.892033 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881df572-1d13-4bce-862b-6f5fb252d8d1 service placement] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:37:26.898458 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c9181-67ff-4137-9b43-0417b12815ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:37:26.899524 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c9181-67ff-4137-9b43-0417b12815ac service nova] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:37:26.899841 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c9181-67ff-4137-9b43-0417b12815ac service nova] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:37:26.900097 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c9181-67ff-4137-9b43-0417b12815ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:37:26.902145 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b67c9181-67ff-4137-9b43-0417b12815ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:26.915892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c9181-67ff-4137-9b43-0417b12815ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:37:26.917527 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c9181-67ff-4137-9b43-0417b12815ac service nova] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:37:26.931381 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1444/2893] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:37:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:37:36.128949 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=34 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:37:36.925746 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=41 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:37:38.107291 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6998667-5cdc-4bd4-ac30-13015b5802da service neutron] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:37:38.121980 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6998667-5cdc-4bd4-ac30-13015b5802da service neutron] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:37:38.128232 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee9b82-976c-458a-bf59-bb329bf3b2a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:37:38.128887 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee9b82-976c-458a-bf59-bb329bf3b2a7 service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:37:38.129173 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee9b82-976c-458a-bf59-bb329bf3b2a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:37:38.129451 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee9b82-976c-458a-bf59-bb329bf3b2a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:37:38.131576 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-e4ee9b82-976c-458a-bf59-bb329bf3b2a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:38.144489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee9b82-976c-458a-bf59-bb329bf3b2a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:37:38.146106 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee9b82-976c-458a-bf59-bb329bf3b2a7 service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:37:38.157342 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1450/2894] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:37:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:37:39.957543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c9181-67ff-4137-9b43-0417b12815ac service nova] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:37:39.959056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-b67c9181-67ff-4137-9b43-0417b12815ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:39.970866 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c9181-67ff-4137-9b43-0417b12815ac service nova] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:37:39.974615 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305040e9-db89-4f76-8657-767c0026e9ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:37:39.975264 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305040e9-db89-4f76-8657-767c0026e9ce service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:37:39.975567 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305040e9-db89-4f76-8657-767c0026e9ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:37:39.975882 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305040e9-db89-4f76-8657-767c0026e9ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:37:39.977635 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-305040e9-db89-4f76-8657-767c0026e9ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:40.040688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305040e9-db89-4f76-8657-767c0026e9ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:37:40.042128 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305040e9-db89-4f76-8657-767c0026e9ce service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:37:40.052342 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1445/2895] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:37:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:37:41.629531 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee9b82-976c-458a-bf59-bb329bf3b2a7 service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:37:41.641529 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee9b82-976c-458a-bf59-bb329bf3b2a7 service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:37:41.647665 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de77dc3-31bc-4353-880f-4cb03b9cd4aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c1966524091242e7a6cd2758027a0d4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:37:41.648374 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de77dc3-31bc-4353-880f-4cb03b9cd4aa service nova] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:37:41.648824 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de77dc3-31bc-4353-880f-4cb03b9cd4aa service nova] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:37:41.649103 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de77dc3-31bc-4353-880f-4cb03b9cd4aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:37:41.655097 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-3de77dc3-31bc-4353-880f-4cb03b9cd4aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:37:41.668811 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de77dc3-31bc-4353-880f-4cb03b9cd4aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:37:41.670502 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de77dc3-31bc-4353-880f-4cb03b9cd4aa service nova] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:37:41.682206 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1451/2896] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:37:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:37:50.047380 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=30 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:37:51.677447 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=12 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:39:34.676557 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305040e9-db89-4f76-8657-767c0026e9ce service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:39:34.716270 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305040e9-db89-4f76-8657-767c0026e9ce service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:39:34.721140 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7f040-ea4e-4edb-8d4f-c6dc64835d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3f887d5bbcd424884537b95affd96f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:39:34.721791 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7f040-ea4e-4edb-8d4f-c6dc64835d86 service neutron] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:39:34.722077 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7f040-ea4e-4edb-8d4f-c6dc64835d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:39:34.722354 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7f040-ea4e-4edb-8d4f-c6dc64835d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:39:34.726944 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-55d7f040-ea4e-4edb-8d4f-c6dc64835d86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:39:34.852729 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7f040-ea4e-4edb-8d4f-c6dc64835d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:39:34.854172 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7f040-ea4e-4edb-8d4f-c6dc64835d86 service neutron] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:39:34.876476 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1446/2897] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:39:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:39:38.189416 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de77dc3-31bc-4353-880f-4cb03b9cd4aa service nova] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:39:38.266950 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de77dc3-31bc-4353-880f-4cb03b9cd4aa service nova] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:39:38.273748 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53abc9-65cf-44e8-9135-fd400dafe675 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:39:38.274385 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53abc9-65cf-44e8-9135-fd400dafe675 service placement] REQUEST_METHOD: `GET` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:39:38.274925 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53abc9-65cf-44e8-9135-fd400dafe675 service placement] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:39:38.275043 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53abc9-65cf-44e8-9135-fd400dafe675 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:39:38.276747 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7e53abc9-65cf-44e8-9135-fd400dafe675 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:39:38.289594 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53abc9-65cf-44e8-9135-fd400dafe675 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:39:38.291617 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53abc9-65cf-44e8-9135-fd400dafe675 service placement] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:39:38.304058 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1452/2898] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:39:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:39:42.127239 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7f040-ea4e-4edb-8d4f-c6dc64835d86 service neutron] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:39:42.131543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-55d7f040-ea4e-4edb-8d4f-c6dc64835d86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:39:42.146127 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7f040-ea4e-4edb-8d4f-c6dc64835d86 service neutron] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:39:42.153892 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:39:42.156230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:39:42.156230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:39:42.156230 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] PATH_INFO: `/v3/users/7d53405e3f88439f81b67106ad98adc2` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:39:42.159113 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d53405e3f88439f81b67106ad98adc2)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:39:42.163609 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:39:42.232466 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d53405e3f88439f81b67106ad98adc2'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.250350 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d53405e3f88439f81b67106ad98adc2'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.251659 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d53405e3f88439f81b67106ad98adc2'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.253072 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:39:42.260965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d53405e3f88439f81b67106ad98adc2'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.261914 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d53405e3f88439f81b67106ad98adc2'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.278165 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1447/2899] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:39:42 2022] DELETE /identity/v3/users/7d53405e3f88439f81b67106ad98adc2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:39:42.493693 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53abc9-65cf-44e8-9135-fd400dafe675 service placement] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:39:42.505889 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53abc9-65cf-44e8-9135-fd400dafe675 service placement] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:39:42.597034 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:39:42.597930 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:39:42.598213 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:39:42.598493 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] PATH_INFO: `/v3/projects/141bd2be69f243aa926f28b96eb6f0d5` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:39:42.599800 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=141bd2be69f243aa926f28b96eb6f0d5)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:39:42.601872 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:39:42.662623 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141bd2be69f243aa926f28b96eb6f0d5', 'request_id': 'req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.668324 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141bd2be69f243aa926f28b96eb6f0d5', 'request_id': 'req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.677993 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141bd2be69f243aa926f28b96eb6f0d5', 'request_id': 'req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.694061 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] The token cache is being invalidate because project 141bd2be69f243aa926f28b96eb6f0d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:39:42.694556 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.695489 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.697814 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1453/2900] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:39:42 2022] DELETE /identity/v3/projects/141bd2be69f243aa926f28b96eb6f0d5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:39:42.703278 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:39:42.705688 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:39:42.728307 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3aa28-27b2-4e19-abc3-375c23ea8125 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:39:42.791091 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:39:42.791983 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:39:42.792265 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:39:42.792543 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] PATH_INFO: `/v3/users/138cb9d2d8884241aed9ebb55c9b814d` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:39:42.793965 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=138cb9d2d8884241aed9ebb55c9b814d)` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:39:42.795977 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:39:42.846699 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138cb9d2d8884241aed9ebb55c9b814d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.858397 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138cb9d2d8884241aed9ebb55c9b814d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.859103 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138cb9d2d8884241aed9ebb55c9b814d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.866933 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138cb9d2d8884241aed9ebb55c9b814d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.867580 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.notifications [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138cb9d2d8884241aed9ebb55c9b814d'} {{(pid=80221) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:42.883343 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1448/2901] 10.222.0.93 () {58 vars in 1223 bytes} [Fri Jun 3 15:39:42 2022] DELETE /identity/v3/users/138cb9d2d8884241aed9ebb55c9b814d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 03 15:39:43.103011 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] Authenticating user token {{(pid=80222) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:39:43.104485 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.fernet_utils [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80222) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:39:43.122559 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7fd7e-29a7-401c-94b9-87ca25315f1d admin admin] Validating token access rules against request {{(pid=80222) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:39:43.175729 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:39:43.189823 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b868f066d8ff4579b72df10988c4ede1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b7e9c80374f4695bdf655157e8cb4d3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80222) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:39:43.190722 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:39:43.191006 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:39:43.191284 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] PATH_INFO: `/v3/projects/d0bc0a82cd0b4587b37873a83f39c3cf` {{(pid=80222) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:39:43.192857 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0bc0a82cd0b4587b37873a83f39c3cf)` {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:39:43.194813 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] RBAC: Authorization granted {{(pid=80222) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:39:43.243720 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0bc0a82cd0b4587b37873a83f39c3cf', 'request_id': 'req-8cab29f1-a071-4071-8125-8cf7a027cd2a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:43.249169 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0bc0a82cd0b4587b37873a83f39c3cf', 'request_id': 'req-8cab29f1-a071-4071-8125-8cf7a027cd2a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:43.257095 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0bc0a82cd0b4587b37873a83f39c3cf', 'request_id': 'req-8cab29f1-a071-4071-8125-8cf7a027cd2a'} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:43.269230 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] The token cache is being invalidate because project d0bc0a82cd0b4587b37873a83f39c3cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80222) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 03 15:39:43.269510 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:43.270161 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG keystone.notifications [None req-8cab29f1-a071-4071-8125-8cf7a027cd2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80222) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 03 15:39:43.271810 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: [pid: 80222|app: 0|req: 1454/2902] 10.222.0.93 () {58 vars in 1232 bytes} [Fri Jun 3 15:39:43 2022] DELETE /identity/v3/projects/d0bc0a82cd0b4587b37873a83f39c3cf => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 03 15:39:52.878813 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=59,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:39:53.266205 dv-cinder-831247-16-iscsi devstack@keystone.service[80222]: DEBUG dbcounter [-] [80222] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80222) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 03 15:40:38.335441 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] Authenticating user token {{(pid=80221) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 03 15:40:38.337002 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:40:38.434676 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e0e652-340b-4ba1-890c-d27a98e4f209 admin admin] Validating token access rules against request {{(pid=80221) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 03 15:40:38.495056 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190385c2-5d17-452a-94eb-65f16e783d45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50625c2b38ed41698ba6cccdb12730ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '23546323ca33412fb388afe308a97499', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80221) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 03 15:40:38.495709 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190385c2-5d17-452a-94eb-65f16e783d45 service placement] REQUEST_METHOD: `GET` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 03 15:40:38.495990 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190385c2-5d17-452a-94eb-65f16e783d45 service placement] SCRIPT_NAME: `/identity` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 03 15:40:38.496263 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190385c2-5d17-452a-94eb-65f16e783d45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80221) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 03 15:40:38.498044 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.fernet_utils [None req-190385c2-5d17-452a-94eb-65f16e783d45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80221) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 03 15:40:38.511461 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190385c2-5d17-452a-94eb-65f16e783d45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 03 15:40:38.513231 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190385c2-5d17-452a-94eb-65f16e783d45 service placement] RBAC: Authorization granted {{(pid=80221) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 03 15:40:38.524415 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: [pid: 80221|app: 0|req: 1449/2903] 10.222.0.93 () {60 vars in 1365 bytes} [Fri Jun 3 15:40:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 03 15:40:48.519338 dv-cinder-831247-16-iscsi devstack@keystone.service[80221]: DEBUG dbcounter [-] [80221] Writing DB stats keystone:SELECT=67 {{(pid=80221) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}